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Chapter I 
Introduction 

1_. General Discussion 

Although still a comparatively young technology, remote sensing of 
the environment has greatly extended man’s perception of the world's 
resources and interaction of natural and unnatural influences. Remote 
sensing has grown from simple photography and photointerpretation to sa- 
tellite borne sensors and sophisticated machine aided analysis. A cri- 
tical portion of many modern remote sensing systems is a mul tispectral 
scanner. Mul tispectral scanner systems employ sensors to observe por- 
tions of the electromagnetic spectrum typically ranging from the visible 
region to the reflective infrared regions. The .thermal (or emissive) 
portion of the spectrum also has important uses in remote sensing. 

Thus, investigation of mul tispectral scanner systems and parameters of 
mul tispectral scanner systems is an important and necessary endeavor. 

Mul tispectral scanner systems are characterized by many parameters 
interacting in complicated ways. This research develops analytical 
techniques for the study of some of these parameters. Many of the 
parameters tend to be dependent on the scenes observed by the mul tispec- 
tral scanner systems. It is thought that consideration of scene depen- 
dent parameters in the current research provides a framework for con- 
sidering very specialized scanner systems as well as more general scan- 
ner systems. An important example of a parameter that must be con- 
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sidered for mul tispectral scanner systems is what portions of the elec- 
tromagnetic spectrum are to be observed. It is clear that this may tend 
to be a very scene dependent parameter. Indeed, under differing obser- 
vation conditions, it may be necessary to observe different portions of 
the electromagnetic spectrum to obtain the desired information about a 
single specialized scene type. The development of analytical techniques 
to aid in the study of some of the parameters of mul tispectral scanner 
systems is the objective of this research. 

2 . Previous Work 

Up to the present, there has been little analytical work aimed at 
general techniques for the study of parameters of mul tispectral scanner 
systems. Most reported work has tended to be ad hoc and empirical. 

This has produced detailed knowledge of various aspects of remote sen- 
sing problems, but has not produced studies of of mul tispectral scanner 
systems in an analytic context. 

Studies such as those by Gates, Keegan, Schleter and Weidner CG33 
and Sinclair, Hoffer and Schreiber CS43 are indicative of the type of 
detailed knowledge that’ has been gained about scenes that may be obser- 
ved by mul tispectral scanner systems. Examples of studies that have 
been conducted for specific problems are those by Coggeshal l and Hoffer 
£C33, and Kumar and Silva CK53. These papers are not referenced so much 
for their contents, but rather as examples of the wide variety of stu- 
dies that have been carried out in an effort to understand aspects of 
remote sensing problems. 

An early study that considers a physical basis for remote sensor 
system design is described by Holmes and MacDonald CH43. This paper 
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gives a good exposition of many of the physical considerations for mul— 
tispectral scanner system design. A more recent study by Landgrebe, 
Biehl and Simmons LL21, Q.41 considers in an empirical manner several 
important parameters in mul tispectral scanner systems. Some of these 
parameters are spatial resolution and spatial sampling characteristics, 
spectral sampling and bands, and signal-to-noise characteristics’. These 
are important parameters and many conclusions can be drawn from empiri- 
cal study. However, it is thought that the development of analytical 
techniques to study some of these and other parameters is now appropri- 
ate. 


_3. The Present Investigation 

As previously mentioned, very little anaytical consideration of 
many mul tispectral scanner system parameters has been done. It is the 
intention of this research to develop analytical techniques to study 
some of these parameters. Although the developed techniques are appli- 
cable to a wide variety of scenes, this reseach uses vegetation scenes 
as a vehicle for consideration of the techniques. 

Consider a single type of vegetation illuminated by the sun. If 
the reflected electromagnetic energy in the visible to reflective in- 
frared wavelengths (approximately .4 to 3.0 micrometers, ym) is measured 
as a function of wavelength, X, for several different observations of 
the same vegetation type, it is observed that the spectral response ex- 
hibits random variation about a mean value at each wavelength. That is, 
the observations tend to be stochastic in nature. Now, if the vegeta- 
tion scene is observed remotely, say from a satellite or airborne plat- 
form, there are additional disturbances of the observations. ' These dis- 
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turbances may be due to atmospheric noise, random disturbance of the ob- 
servation platform., or other sources. The point is that the multispec- 
tral scanner system receives electromagnetic energy from the scene that 
exhibits random variations corrupted by disturbances that also have ran- 
dom variation. Under these conditions, it is logical to suppo.se that 
certain spectral regions (or spectral bands) may be more useful than 
others for observing those features of the scsene. that may be of in- 
terest. It is also logical to conclude that by studying the effect of 
the disturbance on the observation of the scene, it may be possible to 
minimize the adverse effects. Thus, in view, of the above comments, it 
would be useful to characterize analytically what information the obser- 
vation conveys about the scene. 

This is highly reminiscent of the classical problem in communica- 
tion systems. A receiver (mul tispectral scanner) obtains a- signal (the 
electromagnetic energy from the scene) that is corrupted in some manner 
(perhaps, by random noise). It is then desired to introduce a quantitive 
measure of what the received signal conveys about the transmitted' sig- 
nal. This is the special scope of the subject of information theory. 

The birth of the field may be said to be, of course, in the work of 
Shannon CS1 □ . There are voluminous references in the field with major 
texts by Fano CF4] and Gal lager CGI □ . The relation of received signal 
to transmitted signal is described in information theory by the concept 
of average (mutual) information. Loosely speaking, the average informa- 
tion in the received signal about the transmited signal may be said to 
be the reduction in uncertainty about the transmitted signal that is ob- 
tained from the received signal . 
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Application of the concept of average information to the stud/ of 
some parameters of mul tispectral scanner systems is pursued in some de- 
tail in this research. It is thought that this gives insight into the 
study of the relative utility of different spectral bands to be used in 
scanner systems for observation of spectral scenes. Further, utlization 
of information theoretic concepts can be used to study the effects of , 
noise disturbances on the observation of spectral scenes. The develop- 
ment of the information theoretic concepts is the topic of Chapter II. 

The computation of average information in spectral data received at 
the mul tispectral scanner about an observed spectral scene is not 
without difficulties. A method to circumvent the necessity of solving 
some rather intractable equations is described in Chapter II. Also, 
methods for digital computation of average information are developed in 
Chapter II. These computation procedures require that models for the 
spectral response of a scene be developed. 

Chapter III contains the development of the techniques used to con- 
struct models for spectral scenes. Several approaches to construction 
of the models could be pursued. Most of the approaches are studied in 
terms of the system identification problem. Saridis has done extensive 
work on stochastic approximation methods for systems identification 
CS51, CS63, and CS73. The stochastic approximation techniques have the 
advantages of being relatively easily implemented and having great gen- 
erality. Maximum likelihood identification techniques have also been 
extensively used. The references by Kashyap and Rao CK33, and Kashyap 
CK63 give good dicussions of the maximum likelihood identification tech- 
nique. The maximum likelihood techniques are used in Chapter III to 
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develop techniques for constructing models for spectral scene.s. Con- 
cepts from the area generally known as time senes analysis are also 
developed for use in construction of models for spectral scenes. Box 
and Jenkins CB1], Anderson CA4D, and Kashyap and Rao QK3D are good re- 
ferences for time series analysis and its application to construction of 
dynamic models from empirical data. Also discussed is a Bayesian iden- 
tification technique for models of spectral scenes. This technique is 
an adaptation of a method described by Kashyap and Rao CK3] to the 
present work. 

A criterion for selecting one of several hypothesized models for a 
spectral scene is discussed in Chapter III. Further, once a candidate 
model for a spectral scene has been selected, the question of the vali- 
dity of the model remains. Validation techniques for testing candidate 
models for spectral scenes are also discussed in Chapter III. 

Chapter IV uses the model construction techniques developed in 
Chapter III on empirical data from actual scene types that may be obser- 
ved by a mul tispectral scanner system for remote sensing of agricultural 
scenes. The empirical data consists of two vegetative scene types. To 
demonstrate the model construction technique on a scene of a single 
vegetation type, a wheat scene is considered. A set of empirical data 
made up of several vegetation types is used to demonstrate the model 
construction techniques on a more general vegetative scene. The empiri- 
cal data sets are divided into several spectral bands for two reasons. 
First, most mul tispectral scanner systems tend to be designed around 
different sensors for different spectral bands. Second, it is thought 
that better models of the spectral scenes can be obtained by considering 
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several bands in the spectral region of interest than if only one model 
is constructed for the entire spectral region (approximately .45 to 2.4 
um for the present case) under consideration. Several models are hy- 
pothesized for each band and the parameters characterizing each are 
identified using the maximum likelihood technique. Candidate models are 
then selected using the selection criterion developed in Chaper III. 
Finally, candidate models are validated using the techniques developed 
in Chapter III. 

In Chapter V a simple application of the computation of average in- 
formation as developed in Chapter II is carried out using the models for 
the spectral response developed in Chapter IV. This application is in- 
tended to demonstrate how the average information computation can be 
used to select a subset of spectral bands. Also it is demonstrated that 
average information can be used to study such parameters as signal -to- 
noise properties in different spectral bands. The relation of average 
information to spectral bandwidth is implicit in the discussion. 

Chapter VI is devoted to conclusions about the research and 
thoughts for extension of the research. 
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Chapter II 

Information Theoretic Approach 

1_. Introduction 

In this chapter, information theory concepts are developed for use 
in the study of spectral scenes. The basis for this study is the manner 
in which the spectral response is considered. The spectral response for 
several observations of the same variety of vegetation exhibits random 
variation about a mean spectral response at each wavelength. Thus a ma- 
jor consideration for representation of a spectral scene is the ability 
to adequately model its inherent randomness. Another consideration is 
analytical tractabi l ity. Hence, it is reasonable to consider the spec- 
tral response of a scene as a sample function of a portion of a random 
process in wavelength. That is, the spectral response is given by sCxV 
where s(x)eS and X e £x ^ , X 2J • The-ensemble of sample functions for the 
spectral response is S and |x ^ , X is the interval of wavelengths of in- 
terest. It is not necessary to assume that the spectral random process 
is stationary. In fact, it will be seen later that the spectral process 
will, in general, not be stationary. On the basis of empirical studies 
by Fu, Landgrebe and Phillips CF1D, a reasonable assumption on the sta- 
tistics of the spectral process can be made. The spectral random pro- 
cess will be assumed to be a gaussian process. The mean and variance of 
the process will be apparent when models of the spectral process are 
discussed in the next chapter. The gaussian assumption can also be jus- 
tified from another point of view. When a mul tispectral scanner system 
views a scene, it receives a signal from many sources in the field of 
view. If it is assumed that the scanner system is observing many in- 
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dependent and identically distributed sources., then the central limit 
theorem can be invoked to justify the assumed gaussian statistics. The 
gaussian assumption is important in the analytical results of this 
research. 


2 . Definition of Average Mutual Informatipns 

The signal received by a mul tispectral scanner is assumed to con- 
sist of the spectral signal for the scene, s(x), disturbed by a statis- 
tically independent additive random process (noise). This noise process 
consists of the disturbances in the spectral scene not attributable to 
the vegetation under observation and random disturbances in the channel 
between the scene and the multispectral scanner. In the present 
research these disturbances are all combined into one nois,e random pro- 
cess in wavelength, n(x). The noise is also assumed to be a gaussian 
random process. This assumption is made for the same reasons as for the 
signal process. It is further assumed that the noise process, n(x), is 
white. In the present context, white noise is a zero mean random pro- 
cess with autocovariance given by 


E[n(x)n(u)3 



X ^ ^ X,U ^ x^ 


( 2 - 1 ) 


where 


ECO is the expectation operator, 

6(*) is the Dirac delta function. 

Thus the spectral process received by the multispectral scanner is 
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represented by 


y(x> = s(x) + n(x> 



( 2 - 2 ) 


It is not necessary that the noise be white. However, for purposes 
'of studying the bands of a mul tispectral scanner and their information 
content, this assumption is sufficient. It is still possible to allow 
the white noise to have a different spectral density level in each band. 
The more general problem of mutual information in time continuous pro- 
cesses with non-white noise is considered by Huang CHI], CH23. 

The problem to be considered first is to define, and later calcula- 
te, the average (mutual) information in the process y(x) about the pro- 
cess s(x). First, however, it is necessary to state some basic and 
well-known results from information theory concerning the average infor- 
mation in one set of random varibles about another set of random varia- 
bl es. 

The averge mutual information in a set of random variables 
= ■Cv.j, Vj, ..., v N > about the set of random variables 
JJ = -Cu^, U£, .. .., u M > is defined by CGI] 


1 (U, V) = / J P uv (u,v) log 


U V 


p uv ( ^ 


P u (u)p w (V) 


dudv 


(2-3) 


where 


P|j = joint density function 

— of the sets U and V 
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P^Cu) = den's-ity function of the set 
Py(y) - density function of the set 

and i* andX represent H-fo'.ld and N-fold integrals over' all the possible 
U V 

values of the members of the sets jJ and’ V;. 

Since the definition' of avera'ge (mutual) information is known for 
random variables, an intuit i vel y pleasing approach is to represent the 
spectral random processes in' terms of random variables and thus apply 
the- previously known definition. This - is the approach of Shannon CS1D 
for the case of band- limited time function's and an infinite- Observation 
interval. It has been shown rigorously by Gelfand and. Yagl-om CG2D that 
this approach leads to a val id definition of mutual information for time 
continuous- processes under almost al l conditions. 

Suppose that there exists a set of random variables- 
S = | s^, th'at uniquely determines and is uniquely determined 

by s(x>. Similarly suppose that there exists a set of random variable-s 
Y = -Cy^yg,. . that uniquely determines and' is uniquely determined by 
the portion of y(x) that contains s(x). Under the assumption of in- 
dependence of Si(X) and n'(x-), any portion of y(X) not represented by the 
set Y is irrelevant to the calculation of the average information in 
y(X) about s(x). Then it is reasonable to say that the' average' informa- 
tion in Y about $. is the same as the average information in y(X) about 
s(x) . Thus we make the following defin i t ion of the average mutual infoi — 
mation in the process y(X) about the process- s-(x) . 



- 12 - 


Ks(x), y(x>> ± I (S,Y) 


( 2 - 4 ) 


That the average (mutual) information can indeed be defined in such 
a manner merits some elaboration. A method to determine S and Y for 
given s(x) and n(x> is needed. First consider such a representation for 
s(x). Assume that the process s(x) has a finite mean square value. 

v 

That is 




e[s 2 (x)] < «, 

Xe 

[ x r x J 

Let $ 

;[ 

<t>..Xx) ; i = 1 

, 2, .. 

. J be a 

tions 

"for 

■the class of 

square 

integrab 

s(x) 

may 

be represented 

on the 

interva 




n 




s(x) = l.i.m. 

E s 

(X) 



n*» 

i=1 

* i 


( 2 - 5 )' 


[ X 1' *2] * 


Then 


, X^ < X < x 2 


( 2 - 6 ) 


where l.i.m. is the limit in the mean defined by 


l im E 
n-*-» 


(s(x) - J2 s.4>.(X>y 

i=1 1 1 


= 0 , 


£ x £ x 2 


( 2 - 7 ) 


The random variable s. is defined by 

i 


s. =f s(x)<t>^ Cx) dx 


( 2 - 8 ) 


The details for such a representation may be found in such texts as CGI] 


and CV1]. 
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Similarly for another set of complete orthonormal functions 
^(X), e 2 <X), .... on [ V x 2 ] the received spectral process y(x) can 
be represented as 

n 

y(x) = l.i.m. y. e(x) , X- _< x _< \ 7 ' (2-9) 

n+® i=1 1 1 


where 


y i 



y(x)e(x)dx 


( 2 - 10 ) 


It is often convenient to choose the set Ie..(x) ; i=1, ... > 'to be the 
same as the set I4.. ; i=1 , ... >. In particular, this is true for white 
noise disturbances. 

Thus, if the sets I(j).(x) ; i=1, ... > and le.(x) ; i=1, ... > can 

t T "I 

be determined, we have sets of random variables Is. ; i-1,2,...> ! and 
Iy i ; i=1,2, ...} that uniquely determine and are uniquely determined by 

s(x) and y(X) respectively in the manner previously discussed. The com- 
plete orthonormal sets t^Cx) ; i=1,2,...> and le^(x) ; i=1,2,...> will 

be determined later in a manner that is relevant to writing an expres- 
sion for average information for the processes s(x) and y(x) . 

Hence, if we define 


S_ = Is, 


s n > 


( 2 - 11 ) 


and 
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Y n = <y V y Z' y n > (2_1 

we can write from the basic definition of average (mutual) information 
given in (2-3), 


I<S ,Y ) = / f PCS ,Y )l 
n' n •i J n' n 


og 


S Y 
n n 


pCS,Y > 
n n 


plS n )p<Y n) 


dS n dY n 

n n 


(2-13) 


where p(S n ,Y n ), p(S n ), and p(Y n ) are the appropriate joint probability 
density functions. 

Since in general n will be countably infinite we write 


I(S,,Y> = lim I (S ,Y ) 
_ n n 

n-*-® 


(2-14) 


Thus we have an expression for I(s(x) /y(x)) • The complete mathematical 
details are given by Huang CH13 and Gelfand and Yaglom CG23. 

The problem now at hand is to translate this definition into a form 
that is useful for the current problem of determining the average infor- 
mation in the received spectral process y ( X ) about the- spectral scene 
s(x). In the next section this problem is pursued. 


3^. Cal cul ation of Average Information 

In this section, appropriate sets of basis functions 
■C^Cx), $ 2 (X), ...> and {e^Cx), e 2 Cx) , ...} are defined. These basis 
functions will, at least in principle, yield a calculation technique for 
average information. First consider the representation for s(x) , 
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' s(x) = 2-1 S-<j>.(X) ,!,<)[< X,. 

1=1 


i T i ’ ' 1 — " — A 2‘ 


( 2 - 1 5 1 ) 


-- Note:- -For -notational "simpl"icity"we "have 'changed' 'from’ 
n 

l.i.m. ^ s .<}> . Cx ) to the- above; 
n-*-® i=1 

The- covariance function of s(X) is defined as 


K s (X,u<) = ECs(x)s(uO 


i /U'e |x <| ,X 


(2-1 6') 


and it is straightforward to show that CPI, p. 4313 


Kr(x,u) < K s (X,X)K s _(u,u) 


(2-17) 


Since we-have- restricted- ourselves 1 to- processes with- finite- mean square 
value, it- fol l ows- that 


x 2 , x 2 

fi f K**(x,u)dxdu. 

x 1 x 1 s 


?' E (s 2 a>] 


dx 


2' 


< ® 


(2?*18) 


That is, the' processes under consideration* al so,- have- square- integrable 
covariance functions: Since we are deal ing with a-* gauss i an- random pro- 

cess, a useful- additional property t'hat-is to be- requi red- of the-basis* 
functions is that the terms s^ andt s-^ > i#-j, be uneorrel ated-i That is-, 

M = a j 5 sj <2 - i9> 


<5 


I « 
1) 


1 / 

0, 


i-j 


where 
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The usefulness of this requirement will be seen later. 

These are the classic requirements for the representation of a ran- 
dom process in terms of a Karhunen-Loeve expansion CV1T, CC1J. In this 
expansion, the basis functions $j(X) are the eigenfunctions of the in- 
tegral equation 

X 2 

a.A.(X> =f K (x,u)<]>.(u)du , X- < X < X-> . (2-20) 

J J S J 1 

X 1 


The eigenvalues of the integral equation are the numbers 
•Ca. j = 1, 2, ...> . Of course, the eigenfunctions 

J f 

■C^jCx)/ j = 1, 2, ..*> have the required orthonormality property 



Another property is that the sum of the eigenvalues is the average ener- 
gy of the process s(x). That is, 

a. . (2-22) 

3 

Since it is assumed that s(x) is from a gaussian random process, the un- 
correlated random variables Ts. i=1, 2, ...} are also statistically in- 

1 r 

dependent. This property is used later. 

Now consider the generation of appropriate basis functions 
-Ce^(x), i=1, 2, ...> for representation of the received spectral process 
y(x). Since it is assumed that y(x) is of the form 
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y ( X) = s(x) + n(\) , < X £ x 2 , (2-23) 

where s(x) and n(x) are statistical l y'independent, we can write the ‘co- 
variance function of y(x) as 

Ky (x>u) =K g (x,u) + K n (x,u); x^ _< x>u _< X 2 - (2^24) 

Since it is assumed also that n(x) is gaussian white noise, its covari- 
ance function, 

N 

K n<x,u) = y - 5(x-u) (2-25) 

is not square integrable.> It" is necessary to consider the ramifications 
of this for the selection of the basis functions -Ce-(X) ; i=1^2,...>. 

Consider first the use of K^(x>u) as a kernel for the integral* equation 
( 2 - 20 ). 


3 j 6 j (X ) =/ 
J J X 


2 N, 


1 


— 6 (X"*u)>0 j (u)’du. 


X 1 .< X £ X 2 


(2-26) 


or 


a^ e^. (x) 



(X) 


X 1 < X < X 2 


C2-27) 


The implication is that equation (2-26) is satisfied for any set of 

N 0 

orthonormal basis functions with corresponding eigenvalues aj = 

This result is a direct consequence of the fact that K n (x,u) is a delta 
function. Hence, we. may just as well use (X) = $j(X) to represent the 
received process y(x). We need to make the following clarification 
‘here. From Mercer’s theorem CV1 , p. 1813 we need a compl ete> orthonormal 
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set -Cij»j(x); 3 = 1 , 2 /.. .} to represent the covariance function of the white 
noise. If K s <x,u) is not positive definite, the eigenfunctions 
•Cij)j(x), j=1, „ . will not form a complete orthonormal set 0/1, p. 1813. 

In this case the eigenfunctions can be augmented by a sufficient addi- 
tional number of orthogonal functions to form a complete orthonormal 
set. The importance of obtaining a complete orthonormal set is that 
such a set can be used to expand any deterministic square integrable 
function. The necessity for assuring ourselves that -C$j(x); j=1,2,...> 
is complete will be clear later. We can be content with the assurance 
that even if K g (x,u) is not positive definite, we can still obtain a set 
■C^ (x>; 3 *= 1 , 2 ,...> that is complete. 

Thus, the integral equation that defines the eigenfunctions and 
eigenvalues for y(x) is 


bj<J>j (X) 



_ N 0 


<t> ^ Cx> +J K s (x,uHj (u)du, £ X < X 2 12- 


28) 


If we use 




(2-29) 


we have the original integral equation (2-20) again. The implication is 
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that we may represent the received process with the eigenfunctions 

N n 

'C'fr.jCx); j=1,2,. and. the eigenvalues -Ca. + ^ , j=1,2,...T. That is, 
we may write 


yU> = 12 y.+ (x) , x. < x < x 
j=1 


(2-30) 


(l.i. m. is implicit here). 

Hence, y(X) is represented by gaussian random variables 
■Cy ; i=1,... .> that are uncorrelated and thus statistically indepen- 
dent. The correlation of y. and y^, i,j = 1,2, is given by 


! [ y i y j ] = (a i 


+ *u 


(2-31) 


The process y(x) uniquely determines Y = -Cy^, y 2 , ...}. By using the’ 
eigenfunctions -C<|>j(X); j=1,...> we have a unique representation of that 
portion of y(x) ih the signal' space of s(X). By the independence pro- 
perty of average information, the portion of y(x) not in the signal spa- 
ce of s(x) is irrelevant to the average information in y(X) about s(x). 
Thus, we have achieved the goal of representing the processes s(X) and 
y(X) by uniquely defined sets of random variables (s. i=1,2,...> and 

{y. i=1,2, — >. These sets of random variables are now used to write 

1 f 

an appropriate expression, for average information. 

The covariance matrix for the random variables fs.; i=1,2,...> is a 
diagonal matrix with the ith diagonal element given by E^s^J = a... 
Similarly, the covariance matrix for the random varrables 
■Cy.,; i=1,2,...T is diagonal with the ith diagonal element given by 
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r 21 * 

E y . = a. + . Also since we noted that the noise process n(x) could 

be represented by {^(x); 1 = 1 , 2 ,. we can define a set of random 

varibles for the noise that have a diagonal covariance matrix with the 
ith diagonal element given by 

It is fairly easy to show that a set of gaussian random variables 
•Cy. = s i + n , 1=1,2,...}, have average information about the set of 
.gaussian random variables -Cs. i=1,2,...} given by CS1, Theorem 16D 

1 r 


I (S,Y) 


_ 1 


log 


det C 


det C 


n 


(2-32) 


where C and C are the covariance matrices of -Cy.; i=1,2,...} and 
y n i 

"n 

■Cn i = j- ; i=1,2,...> respectively. Since and are diagonal, we 
can write 


00 N n 

det C = n (a. + 

y i=1 3 c 


and 


N n 

d«.c n =nj2 . 

1=1 


Thus, we can write 


det C •*> _ 


Hence, the average information can be written as: 


(2-33) 


(2-34) 


(2-35) 


A 
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I(S,Y) = - £ *09 

2 j=1 

The average information is now written in a form such that calcula- 
tion may be carried out in principle. Furthermore, the average informa- 
tion can be approximated by using only the first n largest eigenvalues 
in the summation. This is reminiscent of the feature selection problem 
in pattern recognition. Thus average information might be useful as a 
feature selection criterion. 

The present formulation of the average information offers insight 
but still is not in an easily calculable form. The next section is con- 
cerned with deriving a more useful formulation for the average (mutual) 
information. This form will also offer more insight into the idea of 
using average information to study parameters of mul tispectral scanners. 

_4 . The Rel ation Between Average Information and the Wiener-Hopf Optimum 
Fi l ter Probl em 

This section will show the relationship between the average infor- 
mation in the received process y(x) about the spectral process s(x) and 
the Wiener-Hopf optimum filter problem. As is well known, the Wiener- 
Hopf optimum filter gives the optimum (in the mean-square sense) linear 
estimate of a process that is corrupted by an independent, additive 
noise process. In terms of the spectral processes of immediate concern, 
we observe 

y(X) = s(x) + n(x) , A.j _< X X 2 , (2-37) 


2a . 

1 + — i 


N, 


(2-36) 


We then pass the spectral process °y(X) through a linear filter to obtain 




- 22 - 


an optimal estimate s(\) of s(x)- This estimation technique may be 
described by the equation 


s(x) = / h(x,u)y(u)du, ^ 


(2-38) 


where h(x,u) is the impulse response of the optimal filter such that 
e£(s(x) - s(x))^j is minimized. It is straightforward to show [VI, p. 
198-2043 that under our assumptions concerning the spectral processes 
y(x) and s(x) the optimum filter must satisfy 


f 


K g (u,v) h(x,v)dv + ^ h(x,u) = K g (x,u), 


'1 


x 1 < X < x 2 
x^ < u < x 2 


(2-39) 


N 


0 . 


where K g (x,u) is the covariance function of s(x) and is the spectral 
level of the noise process. This above relation is a form of the famous 
Wiener-Hopf equation. 

It is now possible to put h(x,u) in a form that is convenient to 
show its relation to average information. Since the eigenfunctions 
<*.(x), i=1,2,...} form a complete orthonormal set, it is possible to 
write h(x,u) in a series expansion of the form 


h(x,u) = ]£ h <j>- (x) 4 > .(u>, 

i=1 1 1 1 


x,, < x < X 2 

X 1 < u < x 2 


(2-40) 


From Mercer's theorem CV1 , p. 1813 we can write 
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K (X,u) = E a-4> (X)<t> • C,u), \ < x,u < X ? . <2-41) 

S *„/j 11 1 I * ^ 


Substitute (2-40) and (2-41) into (2-39.) to .obtain 



= E a . 4 > (x)$.(u) . (2-42) 

i=1 1 1 1 


Using the ortho, norma l ity -property .of the eigenfu, notions, the a,bove equa- 
tion can .be rewritten as 


E h Ca + ^) $ (xH <u) 

r~L ii 2 ’i i 

i=1 


E a >.(X)$.(u) 
r-i ii i 

i=1 


It is seen that if 



i=1,2,... 


(2-43) 


(2-44) 


then the equality of equation (2-43) is evident. Hence, expand b(x,u) 
in a series as 

co 

h(x,u) = E 
i=1 

This representation of h(X,u) is found to be -useful in relating the 
-Wiener filter impulse response to the average information in y(X) about 


a . 

i 


N r 


x^ < x < x 2 


(2-45) 


a i + r*| 


X^ < u < X 2 
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s(x). 

S. 

We first manipulate some of the basic equations into a useful form. 
The equations to be considered are reproduced below for easy reference. 

X 2 

a.*.(x) =/ K (x,uH . (u)du , X,. _S * £ X ? (2-20) 

) J . s 3 J c 

X 1 

/ 4)^(X)dX = 1 (2-21) 

*\ 3 


I (S/Y) =1 E log 


j=1 


2 a.' 


(2-36) 


Note that in (2-21), ^2 = X 1 + * s ^ nce our major interest is in the 
spectral response interval 1 = x 2 - X^. The first manipulation is \ 
differentiation of I(S,Y) with respect to 1 . This is 


dI(S,Y) _ 1 V' d_ 
di ~ 2 j =1 dl 


logCI + £- 

N o 



„ °° 

4 E ci + 

2 j=i 





da 

) 

di 


or 
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d3(S,Y) _ 1 JZ 

d * ' 2 j=1 


i_^!i 

Nq dt 


1 + fT a i 
lx 0' J 


C2-46*) 


da. 


From the above equation it is seen that an expression for is needed. 
In-order to obtain this derivative, first multiply both sides o.f (2-20) 
by <J>j(A)and integrate over the interval X.j, + 5.J . . This gives 


X +£ 

r 2 c 

] ^ * j a)dx ~j 




x 1 +Jt 


7 * K (X„u)<j> . (u) du 
*\ s ) 


$ (x)dx 


(2-47) 


and using the normalization criterion (2-21) we obtain 


. =/ 1 

J J x 1 


X.+Jt 


I x 1 +)l 
f 1 K (x,u)<t> (u)dui 
s j 


<b . Cx)dx 
J 


(2-48) 


Now take the derivative of (2-48) w.ith respect to 1 . 


da^ -X^+£. 

~ J \ 


d£ 


‘1 


K s (X,X 1 +JO-t>^(X 1 +fi.) + 


X / 3*.(U). 

+ / K s (l ' u) -Jr- du 

x 1 


J 


(X ) dx 


tf ,n 

*1 


K s (X 1 +£,u)4>j (X^S.) + 
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fXAl 3*. CX) 

+ J K (X/U) — * dX 


X 1 s 


3* 


a . ( u)du 
3 


This can be simplified to give 


da. ..X +£ 

a* 1 = 


du 


(2-49) 


**K' 


x,+* 


r 


X.,+* 

1 


K (x,u)<j> . (u)du 

X-i S J 


3(t> . (X) 
3 

3t 


dx 


(2-50) 


We now make the observation that equation (2-20) can be written as 


f x n 

a .a . (x„+&) ~J K (x - +Jt,u) * .(u)du 
3)1 x 1 s 1 ' y j 


(2-51) 


Using equations (2-51) and (2-20) in (2-50) we obtain 


da. 

dt 


— - 2a.$7(x.,+t) 

J 3 1 , 


/•X.+t 36. (x) 

+ 2 a. J, <t,.(x) — 3 


3 X^ T ) 


3)1 


dx 


(2-52) 


A simplification of the integral expression on the right hand side of 
(2-52) is still needed. A useful expression may be obtained from the 
normalization expression (2-21). Differentiate (2-21) with respect to 


£. 
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x 1 +Jt 

X.„ 


4ij(X)dX = 0 


(2-53;) 


or 


2 y*X>~^£ 34>»*(X) 

0 = 4>J- Cx 1 +&) + 2 J 4. j <X) — ij — dx (2--5'4) 

The integral term in (2-54) is the same as in (2-52). Hence., using 
(2-54) in (2-52’) we' obtain 


or 


dT = 2a - a.^C x^n) 


(2-55) 


u a . 

a.^U.,**) . (2-56) 

da 

Thds is the expression for that is nee.ded in equation (2-46).. Mak- 
ing. the appropriate substitution we obtain 


dI.CS,Y) 

31 


j=1 


st'VjV 0 


(1 + 


a.-) 


J. 


(2--57) 


Now compare this expression with the expression obtained for the Wiener 
optimal filter response (2-45). It is clear t-hat the relation between 
(2-45) and (2-57) is 

\ * h(X 1 +a, Xl +&) . (2-58) 
Thus it is seen that since l varies from 0 to X-> - we* may make a sim- 
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ptifying change of variables and write 


I(S,Y) 



( 2 - 59 ) 


Thus we have a simple relationship between the average information 
in y(x) about s(x> and the Wiener optimum filter. The expression h(X,X) 
is the weighting that should ’be given to y(x) at wavelength X in order 
to obtain the optimum mean-square estimate of s(x) at wavelength x. It 
is interesting that there is a relationship between the mean-square es- 
timation of s(x) from the observation y(x) and the expression for 
average information as simple as the on$ given above. 

There are, however, major drawbacks in the relations just derived. 
The first major problem is that the covariance function K s (X,u) must be 
known in order to solve the Wiener-Hopf equation in an analytical man- 
ner. In general the covariance function of a spectral scene is not 
known in analytical form. An estimate of the covariance function can be 
made, but this estimate may not be in a useful analytical form and may 
not be positive definite. Furthermore, estimation errors may add to the 
difficulties of discerning a useful analytic form for K g (x,u). The 
second major problem lies in actually solving the Wiener-Hopf equation 
even under the assumption that a functional form for K s <x,u) is known. 

If the spectral processes are stationary and posess rational power spec- 
tral densities, then the Wiener-Hopf equation can be solved. However, 
stationarity cannot ~ecess3 r ily be 5SSw-ec *or fr see: fa! processes. 
The solution of the Weiner-Hopf equation for nonstationary covariance 
functions is considerably more difficult. 
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5_. Computation Considerations 

the above considerations indicate that a more useful technique for 
determination of the Wiener filter impulse response is needed. A tech- 
nique that is useful for this computation may be found by examining the 
relationship between Wiener filter theory and Kalman filter theory. 

These two theories are really different viewpoints of the same problem. 
Of the two, Kalman filtering offers much more computational capability. 
The relationship between Wiener and Kalman filter theory was shown by 
Kalman and Bucy [KID and Kalman [K2]. 

Since the purpose of this section is to study computation techni- 
ques for average information, and computation is most easily carried out 
in discrete form, we shall first recast the previous expressions in a 
discrete formulation. That is, we shall study the problem in terms of 
discrete wavelengths rather than continuous wavelengths. Thus, the 
spectral process 


y(x) = s(x)+n(x> x 1 x 2 

is written as 

y(k) = s(k) + nCk) kefx^, x^j (2-60) 


where k is an integer corresponding to a discrete wavelength in the 
wavelength interval of interest. The white noise process n(x) then 
becomes a sequence, n(k), of independent, identically distributed zero 
mean gaussian random variables with variance . Next, consider the 
description of the spectral process s(x). Since we have assumed that 
s(X) is a gaussian random process, a reasonable model for s(X) is a 
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linear dynamic system driven by an independent gaussian random process. 
That is, the process s(x> is assumed to be described by a linear vector 
state variable form. The linear vector state variable form is written 
as: 

• 

s^(x) = A^jsCx) + Bw(x) (2-61) 

\ 

where 


s(x> = 

and s.(x), s-<x), ..., s (x> are the state variables. More discussion 
\ c n 

of the state variables is given later. 

A is an (n x n) matrix 


s ( x) = s(x) = 




B is an n x 1 matrix 
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w(X) = an independent gaussian random process 

n is order o f the state variable system- 

The concepts of state and state variable formulations may be reviewed in 
Schwartz and Fnedland CS2, Chapt. 23- Kalman CK23, and Schwartz and 
Friedland C$2, p. 125-1273 also indicate the manner in which a system 
given by (2-61) may be written as a discrete-time dynamic system. By 
analogy the discrete form of the spectral process s(X) is given as: 

s_(k+1) = frs (k) + rw(k) kejx.j, * 2 ] (2-62) 

where 


sOk+1) = 


is an Cn x n) matrix. 

_r is an (n x 1) vector. 

wOk) = a discrete inependent gaussian random process with zero 
mean and variance V w (k-)6 Ok-j ) 

k is an integer that correspond's to a discrete wavelength 

x 2 J - With this representation for s(k) we can write the discrete 
form for y(x) as 
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y(k) = H^s(k) + n(k) 



(2-63) 


where 

H^s^(k) = s(k) is the relationship between the state variable for- 
mulation and the spectral response s(x) for the x that 
corresponds to k. 

This discrete form for representing the spectral processes is much 
more amenable to digital computation than the Wiener-Hopf form. It 
should be recalled that the solution of the Wiener-Hopf equation yields 
the optimum (in the minimum mean-square sense) filter for the estimate 
§(x) of s(x) . The estimate thus may be written 


s(x) -f h(x,u)y(u)du, 
J X 1 “ 


x 1 < X 


< X. 


(2-64) 


The Kalman-Bucy estimation equations can be derived from (2-64). 
Furthermore, the same equations can be derived using other techniques. 
Thus the equivalence of the Wiener-Hopf techniques and the Kalman-Bucy 
techniques have been firmly established. The reader is referred to Sage 
and Melsa CS3, Chapter 71 for details. In addition, the complete Kalman 
filter algorithm is included in Appendix I for reference. In these 
derivations, it is a matter of course to obtain the equation for hU,x) 
as 

JnCx,X> = V (X) • H • R _1 (x) (2-65) 

s 


where 
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s(x) = s(x) - §Cx> 


( 2 - 66 ) 


\l_ (X) -* Var 
s 



= Var 



(2-67) 


and 


ROX) = Var[n(X)]; . (2-685 

In the present research, .jUx) is a scalar. The' Kalman filter algorithms 
provide a natural and efficient technique for the computation of the es- 
timation error variance \/. (x). The algorithms given above are the same 

£ 

for the discrete case when the appropriate substitutions are made in the 
relevant variables. The results are* given' bel-ow 

h^k,k) = V-(k). • H • R -1 (k) (2-69) 

£ 

where 


ke-Cx: X 1 < X < X 2 > 


are- integers corresponding to discrete wavelengths o-f interest, 


s (k) = s(k) - s(k) 


(2-70) 


VJk) = VarjjKk)J = Varj£(k) - £( k r )^j 


(2-71) 


and 
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jHk) = Var[ n (k)] . (2-72) 

It should be noted that h_C k , k > is in vector form to conform with 
the state variable models used in the Kalman filter algorithm. The re- 
lationship between _h(k,k) and h(k,k) can be discerned from the state 
variable signal model. This topic will be covered in more detail in 
Chapter III which is concerned with modelling the spectral process. 

Thus we are able to use Kalman filtering techniques to compute 
h(k,k) and hence average information. The average information in y(x) 
about s(x) is given by 

I (S,Y) = j r- 12, h(k,k) (2-73) 

ke |X-|^ *2 

The Kalman filter computation technique has several advantages over 
the Wiener-Hopf approach. The most obvious advantage is the digital 
computer compatibility of the Kalman filter technique. The Wiener-Hopf 
equation is easily solved in only those cases for which the analytical 
form of K s (x,u) is fairly simple. For other cases, solution of the 
Wiener-Hopf equation ranges from difficult to extremely difficult to 
solve. The second advantage is that it is not necessary to have expli- 
cit knowledge of the form of K<.(x,u) in order to use Kalman filtering 
techniques. This obviates the need for estimation of K s (x,u). Another 
advantage is that n(k) need not be "samples” of a white noise process. 

It is possible to use noise models that have a linear dynamic structure. 

Thus, noise models with nonwhite power spectral densities may be im- 
plemented . 
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The major remaining question is the method by which the signal 
model js(k+1) is obtained. Specifically, it is necessary to obtain the 
matrix 4> and the vector _T_ . If the spectral process s(X) is stationary 
and has a rational power spectral density, then and may be deter- 
mined from knowledge of K s (A,u) CV1, pp. 516-5263. In many physical si- 
tuations K^Cx^u) is not known. The parameters and r_ must then be es- 
timated from whatever empirical data is at hand. This is a problem 
which -has been studied extensively in the area of system identification. 
The use of these techniques for modelling the spectral process s(A) (and 
hence ^(X)) is the topic of concern for Chapter III. Thus we will leave 
this problem for later consideration. 

A* further considers t ion of the Relation Between Average Information 

j 

and Optimum Mean-Square F 1 1 te r i ng 

In this section, the relationship of optimum mean-square filtering 
to average information is considered in a somewhat more direct manner 
than in the previous sections. The formulations are in the state varia- 
ble viewpoint in order to be consistent with the final approach to 
average information computation in the previous section. We shall 
specifically be concerned with showing that the estimate J^k) of s^k) 
given by 

/Y k J (2-74) 


£(k) = E £ k 


where 
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Y k = Cytk) 



(2-75) 


= the observed spectrat process. 

/ 

is the estimate such that the average information in £(k) about 
s^k), I(s(k), _s(k>) is maximized. Then since 

s(k) = ji T _s(k) (2-76) 

and 

s(k)' = H_ T j(k> (2-77) 

we have the estimate s(k) of s(k) that maximizes the average information 
I(s(k), s(k)). 

The estimate s(k) is a natural result of the Kalman filtering tech- 
niqe. Thus, a by-product of the computation for I(S,Y) is the estimate 
£(k) . 

In order to demonstrate the above statements, it is first necessary 
to develop some intermediate results. We first show that the average 
information between the estimate s(k) and the estimation error 
sXk) = ^(k) - _§(k) is zero. That is, 

I(|(k), T(k)) = 0 . (2-78) 

Now I(js(k), s^(k)) = 0 if and only if js(k) and T(k) are statistically 
independent CGI, p. 24D, But, based on our initial definitions and as- 
sumptions, §_(k) and T(k) = ^(k) - J^(k) are gaussian random variables. 
Hence, it is sufficient to observe that s(k) and T(k) are uncorrelated. 
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This is easily seen from 


However, 


E ^ k)T i< k >] = 


E fjr Ck>T /Y J = E^(£(k) T -£(k) T )/Y k J 


= E^(k) T /Y k J - £(k) T = 0 


(2-79) 


Hence 

E [± (k)T A (k) ] = 0 • 

Thus ]T(k) and j[(k) are uncorrelated and hence independent. Therefore, 
it is clear that (2-78) is true. 

Next, some entropy relations are needed. The relations to be shown 

are 


H(s_( k) /j[(k) ) = H(j;(k)/§<k)> = HC?(k)> - (2-80) 

First consider the random variables 2 = _s(k) - _§(k) and = js(k). The 
density function P^ w (_z , w) is to be determined in terms of the density 
function Pgg(s(k)J(k)) . It is easily shown CPI, p. 204D that 

p(_s(k) - §.(k), s(k)) = P Z y(z,,w) = 

= Pjg^+w^^) - P<»j>(,s(k) ,J5(k) ) . (2-81) 

Thus, since s"(k) = s(k) - s(k) we can write 
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p(£(k),jKk)) - p(£(k) - js(k), £ (k)) 

= p(?(k), £(k)) . (2-82) 

So we have 

p(_s(k)/j,(k)) = p(£(k)/§U)) 

and hence 

H(^(k)/s(k)) = H(?(k>A§(k)) (2-83) 

Now we have previously shown that 

I(l(k), rck)> = 0 (2-78) 

Hence 

0 = I(s(k), ?(k)) = n(s'(k)) - HCs(k)/s(k)) 
or 

H(s(k)) = H(|(k)/§(k)) . (2-84) 

Now, we can write the average information I(£(k), £(k)) as 

I(s(k), §Ck>) = H(^(k)) - H(s^(k)/^(k)) . (2-85) 

But using (2-83) and (2-84) in (2-85) we have 

I C_sCk) , s(k)) = H(£(k)) - H(r(k)) . (2-86) 

This is a very useful result. It shows that the average information in 
the estimate _§(k) of the state ^(k) of the spectral process is directly 
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related to the entropy of the estimation error ^(k) = jKk) - ^(k) . Sin- 
ce for a given observation the state ^(k) is fixed, it is clear that to 

i 

maximize Kj^(k), JKk)) it is sufficient t.o m.in.imize HCs^(k))-. 

Now since s^(k) is gaussian, it is straightforward to compute the 
entropy H(s(k)) as 

H(sU» = j log(2ire) n |P k l (2-87) 

where 

|P^| = det ^E^(k)^s T (k)JJ . 

Hence it is clear that to minimize H(s(k)) it is sufficient to minimize 

P. . Tomita, Omatu and Soeda CT1D show directly that this is accom- 
— k 

pi ished by the Kalman filter technique. It is sufficient for our pur- 
poses to note that we already have used the Kalman filter algorithm and 
it is known [S3, Chapter 7D that it gives the minimum error variance es- 
timate for our case of assumed gaussian statistics. 

Thus, it is seen that the Kalman filter algorithm produces an es- 
timate s(k) of the spectral response s(k) that is optimum in terms of 
average information-. The optimum mean square estimate s(k) is thus a 
natural by-product that is consistent with the concept of using average 
information to study parameters of mul tispectral scanners. 

In conclusion, this chapter develops the notion of average informa- 
tion in the received spectral process y(X) about the reflectance spec- 
tral process s(X). Furthermore, a technique for computation of average 
information has been developed. The relationship between optimum mean 
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square estimation and average information for the current problem is 
also shown. 
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Chapter III 

Model Identification, Selection, and Validation Techniques 
J_. Introduction 

This chapter is concerned with finding analytical models that ade- 
quately represent the spectral response process of scenes observed by 
mul tispectral scanners. A major requirement for the models is compati- 
bility with the computational techniques discussed in .the previous chap- 
ter.' Specifically, we are interested in obtaining the necessary 
parameters to represent the, models in the state variable forms discussed 
in Chapter II. In Chapter I, the division of the reflectance spectral 
response into bands is discussed. The technique for constructing models 
must, therefore, be sensitive to different characteristics of the spec- 
tral response process in different -spectral bands. Hence, the techni- 
ques developed in this chapter are motivated by the above constraints. 

Very useful techniques for model construction can be drawn from the 
subject area generally known as time series analysis. References for 
time senes analysis are numerous with major works by Anderson CA43, Box 
and Jenkins [B13, and Kashyap and Rao [K33. The reference to time is 
generally a misnomer in that time nerety represents an indexing varia- 
ble. We shall, of course, use wavelength as our indexing variable. We 
first discuss the models that are used in this -research to represent 
spectral response process of scenes. 
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2 _. Models used to Represent Spectral Scenes . 

Recall that the spectral response of a scene is being considered as 
a portion of a realization of a stochastic process in wavelength. Thus, 
the form of the models used must reflect this consideraion . The models 
considered in this research are stochastic difference equations having a 
general form given by 

ml • m2 

SCk) = £ a SCk-j) + X b *(k-j) + w(k) (3-1) 

j=1 3 j=1 3 

where 

S(k) is the spectral response at the discrete wavelenth k. It 

is gaussian with mean and variance determined by the parti- 
cular structure of (3-1). 

w(k) is a zero mean independent gaussian disturbance with vari- 
ance p. 

i|»(k-j) is a deterministic trend term used to account for certain 
characteristics o.f the empirical data. An example is 
«|i(k-1 )=1 .0, which could be used to account for a nonzero 
mean in w(k) . 


a 


and 


are unknown constant coefficients to be determined. 


ml and m2' are constants that determine the dependence of S(k) on 
preceedmg values of the process. 

Thus the dynamic nature of the spectral process S(k) is expressed in 

/ 

terms of is own values at lower wavelengths, some possible deterministic 
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characteristics, -and a gaussian independent disturbance. This model 
formulation though somewhat restricted is sufficient for the purposes of 
this research. 

For completeness, we shall now introduce some asumptions on the 
model given by (3-1). Since (3-1) represents a linear system, it is 
fully described by its second order statistical properties. The coeffi- 
cients a. and b^ are said to be identifiable if they can be determined 
from a semi-infinite set of observations { S Ck) ; 1 < k < “>} such that the 
difference equation (3-1) uniquely describes the second order properties 
of the observed process S<k). We shall now state some assumptions that 

are necessary and sufficient conditions for the identifiabil ity of the 

/ 

coefficients a. and b . The question of identifiabil ity is covered in 
J J 

detail by Kashyap and Rao CK3, Chap. 43. The assumptions that are used 
in this research are listed below. 


Assumptions 

1) w(k), k = 1,2,... is a sequence of zero mean identically distribu- 
ted, independent gaussian random variables with variance p. w(k> is 
independent of S(k-j) for all j >1. 

2) Define the unit del ay' operator D by: 


Dy(k) = y( k-1 ) . 


then (3-1) can be written as 

1 


S(k) 1 


ml 

E 

j=i - . 


m2 

22 b.«l»(k-j) + w(k) 

j=1 ; 


(3-2) 


The assumption is that all the zeros of the expression 
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ml 

A(D) = 1 - E a.D 3 

3=1 3 

, u 

lie outside the unit circle in the complex plane. This assumption 
gives assurance that (3-1) is asymptotically covariance stationary. 
The relation of this assumption to covariance stationarity is ex- 
plored in detail by Box and Jenkins CB1, Chap. 3D. 

3) Suppose we have several trend terms denoted by. i|> . (k) ,i=1 ,2, . . . 1 . 
Represent these trend terms by the vector 

jk(k) = Ci|i,| (k) , . . . ,i(i^(k) D T . 

1 N T 

Then the assumption is lim-i- E 1»<kH(k) exists and is positive 

N-« N k=1 

definite. This assumption gives assurance that the coefficients of 
most trend terms can be identified. Note, however, that this as- 
sumption is invalid for the useful linear trend <|i(k)=k. Therefore, 
a weaker assumption that follows from the above may be useful. 

4) The vector of trend terms obeys 


E E = - (3-3) 

k=1 i=1 1 1 


J 

for nonzero oi = Co^ ,a.,, . . .n^D . The notation used in (3-3) means 
that the summation diverges. It is now demonstrated that (3-3) fol- 
lows from assumption 3. Consider first 


I lm 
N ► n 


N 

E 

k=1 


; (k) ;> (k) 


T 


ORIGINAL PAGE IS 
OF POOR QUALITY 
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1 N 
1 y 

N k=1 


*-Ck)‘ 


- I im 
N +0 ° 


N 

1 ^ f . . i Vj <li^ (k)»|> (k) 

* ^ \ (k) Ck) ,».., N 1 l 

s Jj *2 (k)2 i5w 


1 

N 


t 


* t (k> 2 


Each of the diagonal terms in the above matrix is obviously positive. 
Hence, by the assumption and the above comment. 




n 

i=1 


l im 
N+“> 


1 

N 


£ -K(k) 2 > 0 
M 


and bounded. Thus for each i, there exists a B.. > 0, such that 



4-?(k) = B. . 
1 1 


Now consider £ 
k=1 

a^b^Q. Hence, 



(a.^.(k)) 2 . 

VI 


There is at least one i such that 


£ £ (a.^.(k)) 2 > £ b 2 4,.Ck) 2 . 

k=1 i=1 1 k=1 1 


But since 
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we have 


l im 
N-*® 


i N 

s E ♦i *” 1 

N k=1 


B. 


lim V f (k)^ = ® 
N+® 1 


Hence, 


co _ 

b 2 £ *.<k> 2 
k=1 


co 


/ 


and the desired relation is shown. 

These assmptions will not be mentioned again unless specific need 
arises. 

We shall now list some types of models that are used in the present 
research. The first type of model is known as the autoregressive model 
of order ml. It is defined by 

ml 

S(k) = £ a s(k-j) + w(k) . (3-4) 

‘ j=1 J 

A second type of model is found to be useful for the case of 
nonzero mean for w(k). This model, called the autoregressive plus con- 
stant trend model of order ml, is given by 

ml 

S(k) = £ a S(k-j) + C + w(k) . (3-5) 

J=1 J 

Note that C corresponds to the coefficient of the trend term 
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*(k-1) = 1.0. 

The third model type is useful in representing a nonstationary pro- 
cess CK3, Chap. 30. This model is in the class of integrated autore- 
gressive models of order ml. We shall have occasion to use two cases of 
this model. The first and more common case is given by 

ml 

VS(k) = X) a. vS(k-j) + w(k) (3-6) 

j=1 3 


where 


V$(k) = S(k) - S (k-1 ) . 

The second case of the integrated autoregres’sive models that will be 
used is given by 


v 2 S(k) 


a V-SCk-j) + w(k) 
j=1 3 ^ 


(3-7) 


where 


V 2 $(k> = S (k) - S (k-2) . 

These models are shown in the next chapter to give good fits to the 
spectral response processes under consideration. The models are also 
easily placed in state variable form. It is recalled from the previous 
chapter that this form is useful in the computational technique used to 
obtain the average information in the received spectral process about 
the spectral response process of the scene. An example for placing- one 
of the above models (3-4) in state variable form may be useful. 
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Exampl e 

Assume that a spectral response is modeled as a third order 
autoregressive process. 

S(k) = a^k-l) + a 2 S(k-2) + a 3 $(k-3) + w(k> 

Define the state variables 
S,Ck) = S (k-2> 

S 2 (k) = S (k-1 ) 

$ 3 (k) = S (k) = a 1 S 3 (k-1) + a 2 $ 2 (k-1) + a 3 S 1 (k-1) + w(k) 

Then we can write in vector and matrix form 



S^k-) 


0 

1 

0 


S 1 (k-1 > 


r 

sm = 

s 2 (k) 

• 

0 

0 

1 


S 2 Ck-1) 

+ * 

0 


S 3 (k) 


_ a 3 

a 2 

a 1_ 


S, (k-1 ) 

-> j 


_1_ 


= S/k-1> + jrw(k) 

and 

S(k) = [0 0 1K(k) = H_ T _S_( k ) . 

We s'hal l next consider techniques for estimating the unknown coef- 
ficients in (3-1). 


3_. Estimation Techniques 

The estimation techniques that will be used for the model types 
discussed in the previous section will have two main properties. First, 
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the form of the estimators depends on the, assumed gaussian nature of 
'w(k). The second major property is that the estimation technique must 
be amenable to computational procedures. That is, it is desired that 
the estimation algorithm be in a recursive computational form. 

Two related estimation techniques are discussed. The first techni- 
que is maximum likelihood estimation which does not depend on prior 
'knowledge of the parameters. The second technique considered is Bayesi- 
an estimation in which prior knowledge about the parameters may be in- 
corporated. The techniques will be shown to produce similar algorithms 
for computation. 

We shall begin with some preliminary manipulations that are useful 
in discussing both estimation techniques. Equation (3-1) is recast in 
the following more compact form. 

S(k) = Z_ T Ck-1 ) + w(k) (3-8) 

where 


Z(k-1) = 


S ( k-1 ) 


S(k-ml) 
( k— 1 ) 


ml 


0 = 


t|i (k-m2) 


b 


m2 
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We shall assume that at wavelength k = N, we have -accumul ated the fol- 
lowing data. 

-- -X-(N-)-= tSCN-) SCI'), ~S(0),..Y;ST-mT)y” ~ "" " (3-9) 



is the portion of the data set from which the estimates are construc- 
ted. is the portion of the data that initializes the dynamics of the 
spectral response process. Furthermore, l et _e be the true parameters of 
the model and let p be the variance of w(k) . 

A) Maximum Likelihood Estimation 

The observations X(N) contain the only empirical data from which 
we can estimate the parameters e and p-. The probability density func- 
tion of the observations is needed to obtain the estimation scheme. 

This probability density can be written as 

P(X(N)> = pCS(N) ,S(-m1 )) 

= pCS (N ) / S CN-1 ) , . , S (-ml) )• p(S (N-1 )/ S(N^2 , SC-ml ) ) 

*...*p(S(1)/S(0),...,S(-m1)) •p(S(0),...,S(-mD) (3-10) 

The likelihood principle states that the estimates of the 
^para'nTe'tens__ 5 ; -and'p-'are'^t 1 ie~>aH-ue-s-o-f— the-parameters that maximize the 
probability density function p(X(N)). This estimate is cailed the full 
information maximum likelihood estimate ( FIML) by Kash.yap and Rao. CK3, 
Chap. 6D. However, the probability density function p(s(0),... 


s( :m1 )) 
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is usually either unknown or in a form that renders (3~10) too complica- 
ted to be useful. Thus, the density function p(S(0),. ..,S(-mD) , which 
represents the effects of the initial observations, will be ignored in 
this estimation technique. If a large number of observations are. 
available, it is reasonable to expect that these initial observations 
will not be of overriding influence. If the maximum likelihood estima- 
tion techniques are applied to the remaining terms of (3-10), we obtain 
the estimates called conditional maximum likelihood (CML) estimates by 
Kashyap and Rao EK3, Chap. 63. The so-called CML estimation techniques 
are used in this research. 

Due to the initial assumption of the normality of w(k), we can wri- 
te for each of the conditional probability densities in (3-10) 


p(S(j)/S(j-1),...,S(-m1)) 




(S(j)-Z T (i-1)e) 2 

p 


(3-11) 


Hence, since we are now only concerned with the contributions of the 
conditional probability density functions in (3-10), we can write as our 
likelihood function 


N 

L(e>p,X(N)) = I] p(S(k) | S C k— 1 ) , - . .,S(-m1 ) ) 
k=1 


or 


N 

" 7 

L(o,p,X(N)) = (2*p) exp 


J (S (k)-Z T (k-1 )p) ‘ 


(3-12) 


k=1 


Since we are considering conditional maximum likelihood estimates, the 
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optimum estimates and p of _e and p are those values that maximize 
the likelihood function L(e,p,X(N)) . Equivalently, the logarithm of the 
likelihood function may be maximized. That is, maximize 

L 1 <e,p,X(N>> = logL(e,p,X(N)) = 


•— log(2ir P ) 


v> (S(k)-z T (k-1) e ) 2 


k=1 


(3-13) 


First obtain the estimate for 0 _ by taking the partial derivative of 
(j 3 ,p,X(N)) with respect to to obtain 


SL^jyp^N)) 3 


30 


30 


^ £ ( S ( k ) - r T ( k H ) £) ( S ( k ) -^ T ( k - 1 )^) 
2p k=1 


-1 

2p 


' N N T 

£s(k)Z(k-1)-£z(k-1 )l' (k-1 )q_ 
k=1 k=1 


= 0 


and equate to zero. 

Thus the estimate becomes 


e*(N) = 


N 


£ Z.( k-1 )_Z T Ck— 1 > 
k=1 


-1 N 

£_Z(k-1)S.(k) 

k=1 


(3-14) 


Next, differentiate (3-13) with respect to p to obtain the estimate p 


3L 1 (0,p/X(N)) .. i 1 N j -> 

_L= = ^1, _L v (s(n-z T (k-i),,) 2 = o 

’ 3 " 2„ Z k=1 


or 
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p*(N) = iZ) (S(k)-Z T (k-1)e_) 2 (3-15) 

N k=1 


Since we wish to estimate p from the observations above, it is necessary 

3L-| 3 L-j 

to solve the equations for and — — jointly to obtain 

3j3_ 3 p 


p*(N) =lZs(k)-Z T (H)0*) 2 (3-16) 

N k=1 


If the second partial derivatives of the likelihood function are taken 
with respect to and p, it is seen that j)_ and p satisfy the optimal i- 
ty criterion. 

For our purposes, it is sufficient that the elements of the matrix 
in (3-14) be linearly independent in k to insure that the inverse ex- 
ists. This is true of all the model structures used in this research. 
Kashyap and Rao LK3, Chap. 43 give a more detailed account of the condi- 
tion under which the inverse in (3-14) exists. Kashyap and Rao CK3, 

Chap. 43 also demonstrate the asymptotic consistency in the mean square 

' £ •£ 
sense of the estimators o (N) and p (N). 

Furthermore, Kashyap and Rao CK3, Chap. 43 show that the asymptotic 

mean and covariance of the estimate e (N) are given by 


ECe*(N)/e, P 3 =e+0 



(3-17) 


where 


Q(x) 

x 


K 4 0 as x 


0 , 


and 
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CovCj}* (N)/jjypD 



£_Z(k-1 )Z. T (k-1 ) 
k=1 


+ c 4> 


(3-18) 


where 

" X — 0 as x h. 0. 

It is also shown by Kashyap and Rao tK3, Chap. 4] that the estimator has 

asymptotically minimum variance and is asymptotically equal to the 

Cramer-Rao lower bound on the estimation variance. 

Next we demonstrate a recursive algorithm for the -computation of 

\ 

the estimate e*(N) in (3-14). The algorithm eliminates the necessity of 
computing the inverse matrix each time a new observation is made. This 
gives a large reduction in ’computational load. 

Let us write 


PCN) 


£_Z( k-1 )_Z T '(k-1) 
k=1 


(3-19) 


Then 


£(N) _1 = £ Z.(k-1>Z T (k-1) + Z;(N-1)Z T (N-1) 
k=1 


or 


PCN)' 1 = P(N-1) _1 + Z(N-1 )Z T (N-1) 


(3-20) 


Now apply the matrix inversion lemma (Sage and Nelsa CS3, pp. 499-500] 
and given in Appendix II) to (3-18) to obtain 
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P/N) = P^N-1 )-P(N-1 )_Z(N"1 ) |^1 +Z T (N~1 ) P_(N~1 )Z/N~1 )J V 7 (N-1 )£(N-1 ) 


or 


PCN) = P(N-I) - P(N-1)Z(N-1)Z T (N-1)P(N-1) 

1+Z T (N-1)P(N-1)Z(N-1) ' 


(3-21) 


Thus (3-21) gives a recursive expression for KN). This expression is 
useful in the derivation of a recursive algorithm for the estimate 
e (N) . First write (3-14) in the form 


e*(N) = P_(N) £z(k-1)S(k) 
k=1 


(3-22) 


Then 


e*(N) = P(N) 


N-1 

£_Z(k-1)S(k) + _Z(N-1)S(N) 
k=1 


N-1 

= PCN) £l(k-1)S(k) + Kn)Z^N-I)S(N) . 
k=1 

But from the form of (3-22) we can write 

0*(N) = P(N)P(N-1) : V*(N-1) + P(N)Z(N-,1)S(N) . (3-23) 

Now inserting the expression for _P(N) from (3-21) into the first term on 
the right hand side of (3-23) we obtain 
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' e*CN) = 


P(N-1) 


P(N-1)Z(N-1)Z T (N-1)P(N-1) 

1+^ T (N-1)KN-1)Z_(N-1) 


P<N-1)”V 


(N-1) 


+ P(N) Z(N~1 )S (N) ' 


or simplifying 


/(N) = /(N-1) - LL ^~l> . z , < . l i - . l )zT<N l l ) 6* (»-1 > . + 

1+Z T (N-1)P(N-1)Z(N-1) 


+ P(N)Z(N-1)S(N) . (3-24) 

Consider the last expression on the right-hand side of (3-24). We can 
write from, (3-21 ) 


P(N)ZCN-I) = PCN-DZ(N-I) - 

1+Z T (N-1)P(N-1)Z(N-1) 


or 


P(N-)_Z(N-1) = P(N-1)Z(N 1) # (3-25) 

1+Z T (N-1)P(N-1)Z(N-1) 


substitute (3-25) into the last expression in (3-24) and we obtain 


e*(N) = 9 * (N“1) * P<N-1)Z(N-1) 

1+Z T (N-1)P(N-1)Z(N-1) 


S(N)-Z T (N-1 ) e* (N-1 )J 


Now it is clear that the coefficient of the second term is- identical to 
(3-25) so that we can finally write 
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e*(N) 


= 0*(N-1) + KNU(N-I) js(N)-^ T (N~1)£*(N-1)J 


C3-26) 


The above equation along with 


pcn) = 

~ 1+Z T (N-1)P(N-1)Z(N-1) 


•( 3 - 21 ) 


constitute the recursive algorithm for the estimate 0 (N). For initia- 
tion of the algorithm, one can use an arbitrary (but reasonable) vector 
0 ^ (0) and a positive-definite matrix P(0). 

Thus we have shown the form of the maximuum likelihood estimators 

'fc » • 

0 and p . In the next section, we shal l consider an estimation techm 
que that will allow use of prior knowledge of the parameters to be es- 
timated. 


B) Bayesian Estimation 

In addition to the previously mentioned ability to consider a 
priori knowledge of the parameters, Bayesian estimation differs from 
maximum likelihood techniques in another aspect. This aspect is the in- 
corporation of the loss function which is a measure of the consequence 
of the error in the estimate of the parameters. The optimum Bayesian 
estimate minimizes the expected value of the loss function (called the 
risk function). The loss function most commonly used for engineering 
work is the quadratic loss function 

L( 0 , 0 *) = (eV) T (eV) 

It is well known CK3D, CS33, CV13, CR1], C F 2D that the optimum estimate 
of 0 under the quadratic loss criterion is given by 
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e*(N) = E[g_/X(N)] . (3-27) 

That is, _e (N) is the mean of the posterior density of given the ob- 
servations X(N). With a few preliminary assumptions,, a form for the es- 
timate (N) is derived. These assumptions are: 

bl) The prior distribution of e_ is normal with mean ^ and covari- 
ance matrix Pqp- 

b2) p is the variance of w(k). 

b3) = which insures that ^ consists of coefficients 

of a difference equation that are independent of initial con- 
ditions X^. 

Using these assumptions we can derive the posterior density of given 
the observations X(N). 



(3-28) 


(3-29) 
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Demonstration 

The assertion is demonstrated by examination and manipulation of 
relevant density functions. We have by assumption 

1 " N( 6o.foP> 
and 


w(k) ~ N(0,P) 


for all k. The posterior density of 9 given X(N) is 


P(j3/X(N)) = 


P(e,X(N)) _ 
P (X (N) ) 


p(x(N)/e)pCe) 
p(x cnT) 


(3-30) 


Consider the first expression in the numerator of (3-30). 


p(X(N)/jO = p(S(N),...S(1), S(O)...S(-m1)/j0) 


X 


1 


X 


2 


This can be written as 


p(X(N)/jj) = p(S(N)/S(N-1),...S(1)),X 2 ,^)‘ 


p(S(N-1)/S(N-2),...S(1),X 2 ,ep* 


•...•p(S(1)/x 2 ,e^)*p(e_/X 2 )‘p(X 2 ) . 


But from (3-8) we can write 


p(S(k)/S(k- 1 ),...,s( 1 >,X 2/< 0 ) = 



- 60 - 


V 


1— expti(S(k)-Z T <k-1>e) 2 ] 
5^ L2p ~ “ J 


The second term in the numerator’ of (3-30) is 


p(e) = 


(2irp Ifni 


p| ) 1/2 CXp [zp ( !"VV ( *V] 


The denominator of C3-30) can be written as 


p(XCN» = p(X 1 ,X 2 > = p(X 1 /X 2 )p(X 2 ) 

Now it is noted that pCX^/X^ does not depend on It does depend on 
Qq, Pq, p, and Z^(k-I). Therefore., pCX^/^) can only take on the signi- 
ficance of a normalization factor for the posterior density. This pro- 
perty will greatly simplify the necessary manipul ations. 

Hence, we can write the posterior density (>3-30) ’from the above re- 
lations as 


p(a/X(N)> = 


ft 

k=1 




-exp 




|^l(S(k)-^ T (k-1 


'c2,„|J 0 |J 1/2 eXP [^ <A '^ ’V'i-v] 

1 

Vx,,/X 2 > ■ 


Collecting all normalization factors as a singlre term we. can' write 
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p(e/X(N)) 



(S(k)-z T (k-1 )£> 2 



(3-31) 


The exponent of (3-31) can be written as 


-1 

2p 


£ (S 2 (k)-2Z T (k-1)eS(k) + 8 T Z(k-1)Z T (H)e) 
k=1 


+ JtfoV ~ 2 io^oi + ^o-^-So 


Rearrange and label some terms to give the exponent as 


.1 


■ i 

Ei< 

k=1 


Z(k-1 )_Z T ( k-1 ) +P^ 


P(N) 


-1 


- 2 


■ * - 

El 

k=1 


Z T (k-1)S(k)+eJp^ 1 


P(N) 


P (N) -1 o 


6*(N) T 


+ E)s 2 (k) + JpTV 

k=1 ^ 


*-1 * T 

From the expressions labeled as P(N> and e_ (N) above, we see that in 
order to "complete the square," it is necessary to add and subtract the 


term 
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-1 


^z'Ck-DSUJ+g^ 


k=1 


1 L (n) 

P_(N) -1 

P,(N) 



[l 


£s(k)Z(k-1)+P^ 


* J.*(,N) r - ■ _ •■■ . ■ »- -- - 

When the operation of "completing the square" is carried out, the ex- 
traneous terms- do not depend on 9 _. Therefore, they also contribute to 
the normalization factor previously mentioned. These manipulations give 
the exponent as 


-1 

2p 



(N)) T P(N) 


-1 




where 


e CN) = P(N) 


EsCkHCk-IHf^j 


k=1 


(3-28) 


and 


PCN) = 


■ i 

El' 

k=1 


Z ( k— 1 )Z_ T (k-i J+Pq 1 


-1 


(3-29) 


Thus the exponent is of Gaussian form. The normalizing factor K<j is 
computed from P(N) by 


K 1 S 


M plf.(N)| 


1/2 


where m - ml + m2 


The determinant |P(N)| can be computed as follows 


P(N) -1 


£ Hk-1 ) Z(k-1 )"*” + p: 1 
k=1 -v 
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= PfN-lT 1 + Z(N-1)Z(N-1) T 


= PCN-I)" 1 ^ + P^N-1)Z > (N-1)Z_(N-1) T J 


So 


P(N) = [i + P(N-1)Z(H-imN-1)^] - V(N-1) 
It is relatively straight forward to show that CF3, p.AO] 


j P(N) | = 


| P (N— 1 ) | 


1 + ZCN-1) 'PCN-I)Z(N-I) 


We note that this is a recursive relation with 


= [l+ Pq ZCOJZCO) 1 ]” 1 ^ 


and 


|_P(1)| 


1 + 


i£ol 

I ( 0 )T fni (0) 


Hence, we can write 


J P_CN> j 


l£ol 


n (1 + Z(k-1) T P(k-1)Z(k-1)) 
k=1 ~ 


Hence, the normalization factor is given by 


(3-32) 


(3-33) 
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1/2 


Thus the assertion is demonstrated. 

* Jc 

The Bayesian estimate (N) may be placed in a recursive form that 
is similar to that formulated for the conditional maximum likelihood 
CCML) estimator. In fact, the derivation of the recursive form of the 
Bayesian estimator is exactly the same as for the recursive form of the 
maximum likelihood estimator. The major difference in the two estima- 
tion schemes lies in the manner with which the initial values of the es- 
timate are chosen. In thp Bayesian- technique we can use .our 
knowledge (or assumption) of the, prior density function. Henqe, logical 
initial values would be 

8* (00 = Sq (3-35) 

and 

KO) = Pq (3-36) 

These initial values along with equations (3-26) and (3-21) constitute 
the recursive Bayesian estimation scheme. 

To summarize, we have two similar estimation schemes which give es- 
timates of the parameter vector _o_ based on, the observation, of the spec- 
tral process $(k)', k=1, ..., N. The technique presented thus far is 
clear. The hypothesized models determine which parameters- are to be es- 
timated. These parameters are then estimated by one of the above tech- 
niques. 


(1 + Z(k-1) T P(kr1)_z(k-1)) 


k=1 


2itp1PqI 


(3-34) 
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v 

The question now arises as to what criterion is to be used to 
select a model to represent a spectral process. Thus, we next consider 
a method for selecting a model from the hypothesized models for the 
spectral response. 

_4. A_ Model Selection Criterion 

We begin the discussion of the model selection criterion by de- 
fining a class of models. Recall that we are dealing with models that 
can be written as 

SCk) = Z 1 (k-1 )e_ + w(k) (3-8) 

A class of models is defined by Kashyap and Rao CK3, p.. 181 □ as the tri- 
ple (f, H, g) where f is the stochastic difference equation (3-8), H is 
the range of values for o_ and ft is the range of values for p. A member 
of the class (f, H, n) is written (f, e, p) . The parameter H is defined 
such that every element of one of its members is nonzero. This means, 
for example, that AR(1) models are in a different class then AR(2) 
models. The classes are said to be nonoverlapping. Thus, given several 
nonoverlapping classes and the empirical data set X = {S(1 ),.. ..S(N)}, 
the problem is to select the class which most likely produced the empir- 
ical data. 

The decision rule for choosing a class of models selected is 
derived from statistical likelihood concepts. These methods are develo- 
ped by Kashyap and Rao CK3, p. 183-1883. A different approach that pro- 
duces essentially the same decision rule is developed by Akaike CA13, 
CA23, CA33. The methods of Kashyap and Rao are followed here. The 
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selection criterion is designed to select from the hypothesized classes 
that class which gives the maximum likelihood of producing the given em- 
pirical observations. Suppose that the empirical data X came from a 
model Cf, jig, P g) where ^ = (j)g, p ) is unknown. Let the probability 
density function of X be given by p(X,^g). Furthermore, assume that 
pCX,^) is a known function of X and Then the log-likelihood func- 

tion of p(X,j tQ ) is given in terms of X alone by the following theorem. 

Theorem (Kashyap and Rao) 

Lot A ^e the maximum likelihood estimate of jjig based on the empir- 
ical observation set X. Then 

EC)in(p(X,_^g)) /$*1 = 


= L + EC0(|U q - /ll 3 )/**] 


(3-37) 


where 


L = x,n p(X,£ ) - 


(3-38) 


and 


n = the dimension of 
$ 


* 0 * 


The proof of this theorem is given in Kashyap and Rao CK3, p. 1843 and 
will not be reproduced here. L is regarded as an approximation of 

S,n(p(X,j^-j)) , the l og-l i kel ihood of the class C with the empirical data 

X. 

This theorem suggests the decision rule: 
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a) For every class of models i = 1,... % , find the maximum 

• * "jUf 

likelihood estimate ^ of assuming ^^eH. using the empiri- 
cal data X- Compute the class of log-likelihood functions L.. 
as 

L = jn(p(X, **)> - n. (3-39) 

I *^1 I 

where n. is the dimension of * . = (9 , p • ). 

i -*-oi — oi' K oi 

b) Select the class which gives the maximum value of among 

{ ^ 1—1 r . . f i } • 

This decision rule is relatively easy to use and allows the simultaneous 
comparison of several classes of models. 

Next, we simplify the form of L in (3-39) for the model types used 
in this research. The log-likelihood function is given by 

Anp(X, 4 *) = i,np(S (N) , . . .S(m1 +1)/S(ml) / ... / S(1) ,£*) 

+ £np(S(m1),...S(1),£*) . (3-40) 

Consider the first term on the right side of (3-40). From (3-8) we can 
write 

I = s,np(S (N) , . . .S(m1 +1 ) / $(m1 ),...,S(1) ,j>*) 


= J,n 


N 

n 

k=m 1 +1 


p(S(k))/S(k-1 ),..., S(k-m1) /js *> 
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= a,n 




^L-CS(k)-Z T (k-1 ) A *> 

_2p* 



= - tn(2if *) - JL I] (S(k)-Z T (k-1)e*) 2 . (3-41) 

2 2 P k=m1+1 

But recall from (3-15) that the maximum likelihood estimate of p is 
given here by 



N 

z 


k=m1+1 


(S(k)-Z T (k-1 ) i ) 2 


(3-42) 


Hence (3-41) can be written 

I = tn(2, p *) - JCN-ml) . (3-43) 

Now consider the second term on the right side of (3-40). Let us ap- 
proximate p($(m-1 ),. . .5(1 )/£*) by considering S(m1 ) , . . .S (1 ) as indepen- 
dent Gaussian random variables with zero mean and variance pg. Thus 

II = tnp(S (ml ) , . . .,S(1 ) ,_£*) 


•* an 


ml 

n p(SCk)) 


k=1 


$ anp(S(k)) 
k=1 
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= — =r- Jin(2irp_) S(k)^ . (3-44) 

d b ^ p Sk=1 

So combining (3:43) and (3-44) we can write 

L = j^n p *-mlj 



+ 


N , n9 

~ -£ * n2 ir 

- ^ „ n 

- -£Hn 

PS 

★ 

“ r 


-P - 





S(k> 



L 


b 


=L a+ L b . (3-45) 

It is noted that since in general m1«N, the term corresponding to 
will not vary significantly from class to class when compared to the 
variation in the term corresponding to L . Thus, for comparing, classes, 

a 

it is sufficient to use the simplified form L in the decision rule. 

Let us now discuss the significance of the various terms in the expres- 
sion 

L a = -Sy 4 in(p*) - ml . (3-46) 

* 

The wip term is a measure of the goodness of fit of the model with, the 
estimated parameters to the empirical data. The influence of the number 
of empirical data points is reflected in N. Finally, the ml term acts 
to oppose the selection of increasingly complex models.. This is a quan- 
titative method of incorporating the principle of parsimony in model 
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selction. Parsimony in model construction is a reflection of the intel- 
lectually appealing notion that a simple, but adequate, explanation of a 
physical process is superior to a more complex explanation. Kashyap and 
-Rao' QK'3, p. 185/ 2T4-2T6 T, Kashyap DC4"]7and' Box and Jenkins CB1, pp. 
17-18U further discuss the idea of parsimony. 

Hence, we have a model selection criterion that is analyticaly 
tractable and applicable to aM the model types used in this research. 
Further, this selection criterion has intuitively pleasing properties. 

5_. Val idation of Models 

The procedure for constructing models for the spectral processes 
from the empirical data is straight forward. First, the parameters for 
the hypothesized classes of models are estimated. Then the selection 
criterion developed in the previous section is used to choose the best 
fitting model for the spectral process. The question that remains is 
how well the' selected model represents the empirical data. Specif ical- 

I 

ly, if the model has a certain weakness, then knowledge of the weakness 
can be used to judge whether the model is appropriate for the empirical 
data. The selected model may also be judged on the basis of whether the 
initial assumptions used to formulate the class are valid. The study of 
these topics constitutes the subject of model validation. If the selec- 
ted model fails these tests, then perhaps another class of models should 
be considered. If the selected model passes the validation tests, then 
it is said that the model is valid for representing the empirical data. 
However, it is possible that a class of models other than those hy- 
pothesized may give a better fit to the empirical data. Thus validation 
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of a selected model should not be considered as an absolutely definitive 
statement. Two approaches to validation are used. 

The first set of validation tests check the initial assumptions 
used in constructing the model. The selected model is usedwiththe em- 
pirical data to implement these tests. The assumptions to be tested are 
(see equation (3-8)): 

1) the noise w(k) is zero mean with constant variance p , 

2) w(k) is independent of w(j) for k t j and S(k-j), j >. 

3) periodicities in the empirical data are adequately modeled. 

To conduct these tests, we will use the residual sequence obtained from 
the selected model and the empirical data. This residual sequence is 

X(k) = S(k) - _Z T (k-1 )jb*, k = 1,...,N . (3-47) 

Thus we are using the empirical data and the selected model to estimate 
the noise sequence w(k). The tests using the data generated by (3-47) 
are called residual tests. 

The second set of validation tests determines whether some relevant 
staistical characteristics of simulated data generated by the model are 
adequately close to the statistical characteristics of the empirical da- 
ta. We will be mainly concerned with two tests: 

a) Comparison of correlograms 

b) Comparison of periodograms 

If the selected model is to be used to produce synthetic data, then it 
may be useful to affirm that the synthetic data have the same trend or 
other features as the empirical data. Since we are interested in the 
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model for other reasons (i.e., to aid in the computation of average in- 
formation), we will be concerned only with the tests of the correlogram 
and periodograms. 

Next we consider in detail the tests mentioned above. We begin 
with the residual tests. 

A) Residual Tests 

Test 1_ - Zero Mean Test 

The first test will be for the assumption that w(k), k = 1, ..., N, 
has zero mean. We can recast this in a hypothesis testing context. The 
hypotheses can be written 
H Q : X(k) = wCk) 

: X(k) = 0 + w(k) 

where w(k) is a sequence of independent identically distributed gaussian 
random variables with zero mean and variance p, p > 0 . The alternate 
hypothesis H 1 has e t 0, -» < e < <». It is well known (see Roussas CR1, 
pp. 292-293]) that, in this case, the uniformly most powerful test for 
zero mean (as the null hypothesis) is given by 

1 1 Cx) | < tiqx accept Hg (3-48) 

|t(x) | >_ ng' reject H Q 

where 



t ( x) 


(3-49) 
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x = h- E x(k) 

N k=1 
and 


(3-50) 


P 



N 

E 

k=1 


(x(k) 


7) 2 


( 3 - 51 ) 


t(x) Is the Student’s t-distnbution with N-1 degrees of freedom and 
hence ^ is chosen such that 

P(|t(x) | < hq I Hq) =1-o 

where a is the level of significance, which is defined as the probabili- 
ty of rejecting Hq when Hq is true. The level of significance is chosen 
to reflect the degree of confidence we wish to place in the null hy- 
pothesis. Thus, wer-have an easily applied test for the zero mean as- 
sumption. 


Test 2 - Serial Independence Test 

This is a test of the assumption that the sequence w(k), k=1,...,N 
is serially independent. The test is discussed by Box and Jenkins LB1, 
pp. 289-2933, Box and Pierce CB23, and Kashyap and Rao CK3, pp. 
209-2103. The test is a goodness of fit test. Since it is a goodness 
of fit test, only the hypothesis 

Hq: x(k) = w(k) 

is defined. The alternate hypothesis is the set of all other residual 
models. Note that Hq is the same as the null hypothesis in test 1. 
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That is, Hg is the class of zero mean white noise residual models of 

fe 

variance p . To implement the test, we define the covariance of the 
residual data at lag k as 


1 


N 


R k = N-k ,4 +1 


^2 x(j)x(j-k) 


(3-52) 


The required test statistic is then 


n<x> 


nl 

(N~n 1 ) X/ (RJ 2 
k=1 K 

(ftg ) 2 


(3-53) 


where nl is chosen to be 0.1N to 0.01N depending on the size of the em- 
pirical data set. If the residual data set is as defined byr Hg, then 
n (x) is (approximately) chi-square (x 2 ) distributed with nl' degrees of 
freedom. This gives the decision rule 


n<x) 


< ng, 

- n 0' 


accept Hg 
reject Hg 


(3-54) 


where ng is computed from 

p( n Cx) >_ ng | Hq) = a . (3-55) 

and a is the level of significance of the test. 

Thus, we'have a test which examines the goodness of fit of the co- 
variances taken as a whole. Furthermore, the test is easily implemented 


on the computer. 
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Test 3 - Cumulative Periodogram Test 

The cumulative periodogram test is designed to check for the 
presence of nonrandom periodic components. In our model construction 
technique, it is assumed that deterministic periodic components in the 
empirical data will be accounted for by an appropriate deterministic 
trend term. Hence, we are interested in detecting any significant resi- 
dual periodicities so that we can adjust our models if necessary. 

This test is described by 8artlett CB3D based on arguments from 
stochastic random walk theory and by Box and Jenkins CB1, pp. 294-2973 
on the basis of similarity with the Kolmogorov-Smirnov tests for distri- 
bution functions (see Hoel CH3, pp. 324-327]). 

We consider the equation 


where 


and 


CN/23 


:(k) = E (a^cosuik + B,sin<jj.k) + w(t) 
j= 1 J ) ) 3 


(3-56) 


= largest integer I,- 


_ 2uj' i-i 

W j “ / J 


rtf 

7 


This equation obviously represents the possibility of periodic com- 
ponents in the residual data. It is noted that if the frequencies 
Uj = are considered, then the frequencies = 2ir(1-j)/N are 

redundant if phase information is not considered. It will be seen that 


phase information is not considered here. Hence, the hypotheses under 
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consideration are* 

H q . xCk) = wCk) 


H^: at Least one of the components 


(3-57) 


V B i 


j -1, 


N 

hd 


is nonzero 


This test is performed differently than the' previous two'tests. First 
compute 


g k 




where 


2 _ 


£)' x (k) cos, 
N k=1 


co j .k 


2 : 


N k=1 


x,(k) s-in u .k! 
) 


(3-58) 


(3-59) 


If x(k) = w(k), a plot of g" k , ( vs., k- would: be~ scattered* about' a' straight 
l ine- between the points, (0,0) and (0 + .5>- V.0) . The* probab-i l ity- that the 
cumulative periodogram lies' between* l ines. parallel to. the - line between 


(0,0) and (.5, 1.0) 
Rao Ck 3, p. 2083) 


at distances' ±x 



is* given by (see c Kashyap and 


£. (-1) j exp r 2 x 2 ] 2 } . 

j=-« 


The parameter x is 1.36 for 0.95 probab.il ity and 1.63 for 0.99 probabil- 
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ity. 

Now even if the model is correct, the residuals will not be exactly 
white since the parameters were estimated from a finite data set. 
However, the cumulative periodogram will tend to indicate those boundary 
crossings that are random and those that are due to gross model defi- 
ciencies. If the boundary crossing are due to model inaccuracies, the 
deviations will tend to be large and constitute a considerable portion 
of the cumulative periodogram. Conversely, if the deviations are small 
and occur over a small portion of the domain of the cumulative periodo- 
gram, they might be attributable to randomness in the residuals. 

Hence, the decision procedure is 

1) plot the cumulative periodogram and the boundaries. 

2) if the plot is within the boundaries accept Hg. 

3) if the plot crosses the boundaries either 

a) reject Hg 
or 

b) if the boundary crossings are not gross consider other 
characteristics (i.e., examine plots of simulated and 
empirical data) to determine whether to reject Hg. 

Thus we have a fairly easily implemented test for nonrandom periodici- 
ties in the residual data. 

This completes the descriptions of the residual tests used in this 
research. The tests are all easily implemented on a computer and are 
straight forward. 
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8) Tests of Statistical Characteristics 

t 

Test 4 - Correlogram Test 

. . Th-is test is -designed- to compare tb“e“ autocovariance - functions of 
the empirical data and synthetic data generated from the fitted model. 
Thfe r technique of comparison of correlorams is discussed by- Kashyap and 
Rao CK3, pp. 211-2123. The estimate of the autocovariance functions 
that are used in this test is given by 

. N 

R(k) = _> £ (S(j) -s'XS(j-k) - 7) k«N (3-60) 

N ~ K j-k+1 


where 


s = ^ E SCj) . 

N 3=1 

This estimate is called- the corel logram by Kashyap and Rao. Oppenheim 
and Schafer C01 , pp. 539-5413 show that the estimate is unbiased and the 
variance is asymptotically zero (as N +«) . Hence, the estimate is con- 
sistent in the mean square sense. 

An estimate of thfe correlogram 1 for the fitted model can be obtained 
by computing- the average of the estimate of R(k> given 1 by (3-60) over 
several independent sets of synthetic data. That is, the estimate of 
the correlogram for the fitted’ model from’ J sets of synthetic data is 
given by 
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(3-61 ) 


where 


ftj(k) is the estimate given by (3-60) for the jth sequence 

* I 

of synthetic data. 

The standard deviation of the estimate given by (3-61) is 


a„(k) 



(fi^k) - R M (k)) 



(3-62) 


The estimate of the correlogram for the empirical data is also computed' 
from (3-60) and is denoted fc(k). 

Kashyap and Rao suggest that if the following relationship is sa- 
tisfied/ the correlogram of empirical data can be regarded as adequately 
fitting the correlogram of the synthetic data. 

R H (k) - 2o H (k) < R(k) < R^(k) + 2o H (k>, k=1, ..., N-1 (3-63) 

However the variance of the estimate (3-60) becomes large as k ap- 
proaches N (see Jenkins and Watts [Jl, p. 181] or Oppenheim and Schafer 
C01 / p. 5403). Therefore, the following relation is suggested. 

R M (k) - 2a M (k) < R(k) £ R^k) + 2o H (k), k « N. (3-64) 

Usually k may be chosen to be approximately 0.1N. If the relation in 
(3-64) is satisfied, then the correlogram of the empirical data can be 
said to adequately fit the (estimated) theoretical correlogram of the 


simulated data. 
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Thus, we have a method of comparing the autocovariances of the em- 
pirical data and the synthetic data generated from the fitted model. 

Test' 5"- Periodogram Test' 

The periodogram test is a qualitative test to compare frequency 
components of empirical and synthesized data. The periodogram has been 
discussed as an estimate of the power spectrum by Oppenheim and Schaffer 
[01, pp. 545-554!!. However, as noted by Oppenheim and Schaffer, and 
Bartlett EB3, pp. 274-2883, the estimate is biased and has a standard 
deviation of the same order as the mean of the estimate. Hence, without 
additional processing, the periodogram is not a particularly good es- 
timate of the power spectrum. Therefore} it is not the intention of 
this test to produce an estimate of the'power spectrum'. 

The periodogram is defined as 

Y 2 (k) = 

where ^ - pp 

SCj) is either the empirical spectral process or the synthetic 
spectral process generated from the fitted' model . 

A plot of the periodogram versus t» k for the empirical and the synthetic 
data constitutes the test. If the periodogram of the synthetic data has 
relative peaks at approximately the same frequencies as the' relative 
peaks of the empirical data, then the fit is said to be adequate. It 
must be said that this test is highly qualitative and will indicate only 
gross defects in modeling any periodicities in the empirical data. 


N 

nr £ S< j )c0s “ k i 


^ 11 
IT .^j 


(3-65) 
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These five tests constitute our validation criteria for a selected 
model. If a model successfully completes the tests, we will say that 
the model has been validated. Once again, it must be stated that some 
class of models other than those hypothesized may give a better fit to 
the empirical data. Hence, the model validation criteria are> useful 
only in a relative sense. 


^5. Summary 

In this chapter, we have developed some techniques for obtaining 
the state variable form models for use in the Kalman filter calculations 
discussed in Chapter II. These calculations are then used to determine 
average information in spectral bands. From these computations, an op- 
timum (in terms of average information) subset of spectral bands is 
chosen. 

The model construction technique developed in this chapter is 
listed below in a stepwise sequence. 

1) Hypothesize several classes of models. 

2) Identify (or estimate) the necessary parameters for each class 
of models from the empirical data using either maximum likel- 
ihood or Bayesian estimation techniques. 

3) Select a class of models using the likelihood selection cri- 
terion. 

I 

4) Use the five validation tests to determine if the selected 
model adequately fits the empirical data. 
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The validation of a selected model is for a paticular set of empirical 
data. The selected model is the best fitting model in a class of 
models. The likelihood selection criterion selects the most likely 
cl“a's's-of models to represent the empirical data. In terms of this 
research, the above may be interpreted as meaning that we do not state 
that the validated model is the model for, say, wheat. Instead, we have 
a class of models Ci.e., second order autoregressive) that represents a 
wheat scene in a spectral band. 

Thus, a very flexible technique for constructing models of spectral 
processes has been developed. 



Chapter IV 


Application of Modeling Techniques 


J_. Introduction 

Tms chapter demonstrates the application of the modeling tech- 
niques developed in Chapter III to empirical spectral response data. 

The modeling techniques are shown for two empirical data sets for pur- 
poses of comparison and demonstration that the techniques are applica- 
ble to different spectral scene types. 

The models developed are used in the next 'chapter to demonstrate 
the use of average information to choose subsets of spectral bands. 

First, however, a description of the empirical data used in this 
research is given. 

2 . The Empirical Data 

The data set for this research is chosen to exhibit several charac- 
teristics. First, the data must be representative of the types of- 
scenes observed by the mul tispectrat scanner systems. Second, the data 
set should be amenable to the techniques being pursued in this research. 
Third, the data should be relatively free of artifacts that may be in- 
troduced in the data collection process. Such data sets are available 
at the Purdue University Laboratory for Application of Remote Sensing 
(LARS). All of the empirical data used in this research is gathered 
with the Purdue/LARS Exotech 20C spec trorad IOmeter CL1D, and was col- 
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lected from some test sites in Williams County, North Dakota. 

Two different sets of empirical data are used to demonstrate the 
techniques developed in this research. The fi_rst. set consists o.f obser- 
vations of wheat scenes and was selected primarily for two reasons. 
First, it is desired to demonstrate the techniques' for a scene that con- 
sists of a single type of vegetation . It is thought that this will 
demonstrate the feasibility of using the developed techniques to analyze 
parameters of a mul ti spectral scanner system for observing a particular 
scene type. Second, there is general interest in 'observing the world 
*wheat crop. Reasons for observing the world wheat crop include estimat- 
ing the world food supply rand detecting pathological conditions. 

The second set of empirical data consists of several vegetation 
scene types. Included in this second combined' empirical data. set are 
oats, barley, grass, alfalfa and'fallow fields. Use of this second set 
of data provides insight in using the techniques developed in this 
research to anal yze' parameters' of a mul tispectral scanner system for ob- 
serving a more general scene. Also study of the second data set pro- 
vides a comparison of results' for two different data sets. 

The data described above is available in ’the data library at the 
Laboratory for Applications of ‘Remote Sensing at" Purdue 'University . The 
specific data used in this study is stored on data tape 3990, and each 
observation is identified by its run number. * The observations (run 
numbers) used -for the wheat dataware listed in Tabie IV-1 . Similarly 
the -observations used for the -combined scene are listed in^Table IV-2. 

The empirical data is sub] ected to ' some ' initial processing to 

•render it mbre useful for the -current study. First, ‘.the data for the 

% 



Table IV-1 


Run Number 
75769000 
757691 00 
75769300 
75769400 
7 5769500 
75769600 
75768700 


Run Number 
75768400 
75768500 
75768600 
75768700 
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. Wheat Scene Data Run Numbers 


Run Number 

Run Number 

Run Number 

Run Number 

75769800 

75770600 

75771400 

75772200 

75769900 

75770700 

75771500 

75774900 

75770000 

75770800 

75771700 

75775000 

75770200 

75771 000 

75771 800 

75775100 

75770300 

75771100 

75771900 

75775200 

75770400 

75771200 

75772000 

75775300 

75770500 

75771300 

75772100 

75775400 


Table IV-2. 

Combined Scene 

Data Run Numbers 


Run Number 

Run Number 

Run Number 

Run Number 

75768800 

75774300 

75775700 

75776200 

75768900 

75774400 

75775800 

75776300 

75774100 

75774500 

75775900 

75776400 

75774200 

75774600 

75776100 
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wheat is averaged over the observations. That is, an ensemble average 
for, each wavelength is taken. It is thought that the resultant average 
spectral response provides a relatively good data set to demonstrate the 
techniques "of t'his'research. the average spectral response for the 
>wheat data is shown in Figure IV-1. Similarly, an average spectral 
response for the combined scene is taken. This average spectral 
response may be considered as an average vegetation scene for the pur- 
pose of this research. The average spectral response for the combined 
data is shown in Figure IV-2. It is noticed in both Figures IV-1 and 
IV-2 that there are two data drop-outs at approximately 1.34 - 1.45 mi- 
crometers (pm) and 1.82 - 1.96 pm. These two data drop-outs are due to 
atmospheric absorbtion of the incident and reflected electromagnetic 
energy-. Thus, these two spectral bands are not useful for the current 
research. 

In order that the study be carried out in a context that is rela- 
tively realistic for mul tispectral scanners, the spectral response of 
the two data sets fs divided into spectral bands. The division is rela- 
tively arbitrary, but each spectral band must contain a sufficient num- 
ber of data points to ensure fairly accurate parameter estimation for 
model identification as discussed in Chapter III. The spectral bands 
for the wheat data are shown in Table IV-3. It has been- noted previous- 
ly that the gaps between bands 7 and’ 8 and between bands 8 and 9 are due 
to atmospheric absorbtion of the incident and reflected spectral ener- 
gy. Similarly, spectral bands for the combined scene -are shown in Table 
IV-4. Thus, the data sets and the spectral bands are defined for this 
study. The next step is to identify models for the spectral bands de- 
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Table IV-3. Spectral Bands for Wheat Scene 


Band Number 

Spectral 

Wavelength Interval 

1 

.4528 

- .5380 

pm 

2 

.5380 

- .6239 

pm 

3 

.6239 

- .7097 

pm 

4 

.7097 

- .8517 

pm 

5 

.8517 

- .9910 

pm 

6 

.9910 

- 1.130 

pm 

7 

1.130 

- 1.344 

pm 

8 

1.446 

- 1.821 

pm 

9 

1.959 

- 2.386 

pm 


Table IV-4. Spectral Bands for Combined Scene 


Band Number 

Spectral 

Wavelength Interval 

1 

.4565 

- .5402 (jin 

2 

.5402 

~ .6246* pm 

3 

.6246 

- .7097 pm 

4 

.7097 

- .8481 'pm 

5 

.8481 

- .9850 pm- 

6 

.9850 

- 1.122 pm 

7 

1.122- 

- 1 .307 pm 

8 

1.451 

- 1 .81 8 pm 

9 

1.967 

- 2.386 pm 
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fined above. 

Identified Model s for the Empi rical Data 

This section discusses the models identified for the specral bands 
of the two empirical data sets described in the preceding section . The 
particular identification techmqe used is the maximum likelihood tech- 
nique discussed in Chapter III. This technique obviates the need for 
assumptions about the prior density functions of the parameters. In- 
stead, some arbitary (but reasonable) assumptions are made on the neces- 
sary parameters needed- to initiate the estimation (identification) al- 
gorithm. A sample copy of a computer program that implements the iden- 
tification al-gorithm in FORTRAN can be found in Appendix III. 

The discussion of the models identified for the two empirical data 
sets is ordered by bands. It is thought that aside from being a logical 
method of proceeding, this will provide a simple comparison of 
corresponding spectral bands for the two empirical data sets. 

A. Band 1_ 

From Tables IV-3 and IV-4 it is seen that this spectral band is in 
the wavelength region of .4528 to 15402 ym for the wheat data and .4565 
to .5402 ym for the combined scene data. The model types identified for 
the wheat scene are the autoregressive, autoregressive plus constant 
trend, and the integrated autoregressive models. Hypothesized models up 
to tenth order were identified. The results are tabulated in terms of 
order and selection criterion as defined by (3-46) in Table IV-5. On 
the basis of the selection criterion, it is clear that the sixth order 
autoregressive model is chosen. The residual variance is defined as the 
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variance of x(k) in equation (3-47) and can be considered as a measure 
of the goodness of fit of the model . The residual variance for the 
selected sixth order autoregressive model is 1.26 x 10 Hence, the 
model is judged to be a good fit to the empirical data. The estimated 
coefficients for the sixth order autoregressive model are given in Table 
IV-6. 

There was some difficulty in validation of a model for band 1 of 
the combined scene. Hence, the integrated autoregressivee model of the 
second kind discussed in Chapter III was identified in addition to the 
three model types identified for the wheat scene. Also higher order 
models were identified for the combined scene. It is thought that the 
more complicated models for band, 1 of the combined scene are necessitat- 
ed by the higher variability of the empirical data as seen in Figure 
IV-2. The identified hypothesized models are tabulated in terms of ord- 
er and selection criterion in Table IV-7. The model with the highest 
selection criterion that also passes all the validation tests is the 
eleventh order integrated autoregressive model of the second kind. The 
other models with higher selection criterion values could not pass the 
serial independance test. Hence, we have an example of the ease with 
which the systematic approach to identification of hypothesized models 
developed in this research allows the examination of alternate models in 
the event of the inadequacy of a candidate model . The residual variance 
of the selected model is 1.46 x 1G~^ which is evidence of a good fit to 
the empirical data for this model. The estimated coefficients for the 
eleventh order integrated autoregressive model of the second kind are 
given in Table IV-8. 
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Table 

IV-5. Identified Models for Band 1 
Selection Criterion 

1 of Wheat Scene 

Order of 
Model 

Autoregressive 

Autoregressive 
plus constant 
trend 

Integrated 

Autoregressive 

1 

366.2 

351.7 

364.3 

2 

374.0 

367.5 

362.5 

3 

376.9 

373.6 

363.9 

4 

379.6 

37614 

366.6 

5 

377.5 

377.3 

367.9 

6 

381.2 

379.4 

368.7 

7 


380.6 

369.3 

8 

379.5 

377.8 

367.1 

9 

377.1 

376.5 

364.9 

10 

377.0 

375.8 

364.4 


Table IV-6. Coefficients for Band 1 Selected Wheat Scene Model 


Coefficient 

Estimated .Value 

a 1 

a 2 

.20828 

.16650 

c 

a 3 

.16368 

■J 

a 4 

.16533 

a 5 

.14243 


.16945 
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Table IV-?. Identified Models for Band 1 of Combined Scene 


Selection .Criterion 


Order of 

Autoregressive 

Autoregressive 

Integrated 

Integ rated 

Model 


plus constant 

autoregressive 

autoregressive 


-trend — - - - 

fi rst-'kind- 

- second 'kind" ~ 

*1 ' ’ 

270.1 

289.2 

3T4.1 

304.7 

2 

332.7 

343.8 

330.5 

315.0 

3 

•350.7 

350.0 

331.9 

325.3 

4 

351.2 

351.5 

331.1 

326.0 

5 

350.4 

351 .6 

331 .9 

328.7 

6 

346.6 

347.9 

330.6 

330.5 

7 

342.9 

346.5 

330.7 

328.3 

8 

340.5 

343 .,5 

329.0 

334.3 

9 

339.7 

348.2 

350.2 

352.2 

10 

346.5 

356.4 

34S.5 

352.0 

11 

354.4 

354.3 

349,4 

353.1 

•12 

352.5 

352.5 

350.2 

356.0 

13 

351.4 

354.1 

349.7 

357.5 

14 

354.1 

352.8 

358.2 

358.0 

15 

354,4 

353.5 

356.6 

355-7 

16 

353.8 



352.9 

17 

352.2' 



350.3 

18 

351.6 



349.4 

19 

349.8 



349.3 

20 

348.8 





Table IV-8. Coefficients for Band 1 Selected Combined Scene Model 


Coefficient Estimated Value 



.031594 

.037784 

.13395 

.089051 

.084141 

.10954 

.10055 

.075585 

•12T25 

.033931 

.064937 
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Ei) Band 2_ 

This spectral band encompasses the wavelengths .5380 - .6239 pm for 
the wheat scene and .5402 - .6246 jim for the combined scene. The iden- 
tified models for the wheat scene are the same types identified for band 
1 of the wheat scene. It is thought that since the empirical data is 
fairly well behaved in this spectral band, the more complicated in- 
tegrated autoregressive model of the second kind is not needed. The hy- 
pothesized models are tabulated in terms of order and selection cri- 
terion in Table IV-9. It is clear that on the basis of the selection 
criterion, the second order autoregressive model is to be chosen to 
represent band 2 of the wheat scene. The residual variance for this 
model is 6.19 x 10 ^ indicating a good fit to the empirical data. The 
estimated coefficients for this model are given in Table IV-10. 

The combined scene empirical data is also fairly smooth in this 
spectral band. Hence, only the three basic models are identified for 
this band. The hypothesized models are listed in terms of order and 
selection criterion in Table IV— 1 1 . It is clear from the table that 
the model selected is the second order autoregressive model. The resi- 
dual variance for this model is 1.72 x 10 ^ thus indicating a good fit 
to the empirical data. The estimated coefficients for this model are 
given in Table IV-12. 

O Band 3 

The spectral intervals in band 3 are .6239 - .7097 pm for the wheat 
scene and .6246 - .7097 gm for the combined scene. The three basic 


models were identified for the wheat scene. The identified models are 
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Table IV-9. Identified Models for Band 2 of Wheat Scene 



Selection Criterion 


Order of 

Autoregressive 

Autoregressive 

Integrated 

Model 


plus constant 
trend 

autoregressive 

1 

541.8 

462.4 

490.7 

2 

560.0 

513.3 

499.4 

3 

553i3 

530.1 

507.9 

4 

542.9 

535.5 

513.6 

5 

534.0 

537.9 

520.0 

6 

526.5 

539.2 

525.9 

7 

518.7 

541.3 

530.1 

8 

511.0 

541.1 

538.0 

9 

505.5 

540.1 

540.6 

10 

498.8 

538.2 

539.4 


Table IV-10. Coefficients for Band 2 Selected Wheat Scene Model 
Coefficients Estimated Value 


.50360 

.50139 



- 35 “ 


Tabl.e 1V-11. Identified Models for Band 2 of Combined Scene 



Selection Criterion 


Order of 

Autoregressive 

Autoreg ressive 

Integrated 

Model 


plus constant 

autoregressive 



trend 


1 

491 .6 

439.3 

473.9 

2 

492.0 

469.4 

- 474.4 

3 

490.9 

479.1 

- 477.4 

4 

4S4.0 

478.3 

477.6 

5 

473.0 

475.9 

478.0 

6 

474.4 

474.4 

478-9 

7 

471.2 

472.3 

477.6 

8 

466.6 

472.3 

479.0 

9 

468.1 

471.6 

479.3 

10 

465.7 

470.3 

473.3 

11 

462.8 

469.4 

477.3 

12 

459.6 

467.9 

474.8 

13 

456.6 

467.2 

472.1 

U 

453.9 

466.0 

469.0 

15 

451.4 

465.1 

465.4 


Table IV-12. Coefficients for 8ancf 

Coefficients Estimated 

a- .50475 

a' .49393 


2 Selected Combined Scene Model 
Val ue 
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listed in terms of order and selection criterion in Table IV-13. It is 
clear that according to the selection criterion the eleventh order in-, 
tegrated autoregressive model is to be chosen. The residual variance 
for this model is 6.00 x TO thus indicating a good fit to the empiri- 
cal data. It is interesting that a higher order integrated autoregres- 
sive model gives a better fit than the best (and lower order) autore- 
gressive models. This may be an indication of nonstationarity of the 
spectral process in this band. The estimated coefficients for the 
selected eleventh order integrated autoregressive model are given in 
Table IV-14. 

The combined scene hypothesized models were the same as for the 
wheat scene. The identified models are listed according to order and 
selection criterion in Table IV-1'5. As seen from the selection cri- 
terion either the tenth or eleventh order integrated autoregressive 
model is to be chosen. The el eventh .order model is chosen here since it 
has lower residual variance and has a sightly better selection criterion 
(if computed to more decimal places). The residual variance for the 
selected model is 1.15 x 10 ** which is indicative of a good fit between 
the model and the empirical data. The estimated coefficients for the 
selected eleventh order integrated autoregressive process are listed in 
Table IV-16. 

_D) Band £ 

The spectral bands consist of the wavelength interval .7097 - .8517 
ym for the wheat scene and .7097 - .8481 ym for the combined scene. The 
same three basic model types were hypothesized for this .band of the 
wheat scene. The identified models are given in terms of order and 
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1 

Table IV-13. Identified Models for Band 3 of Wheat Scene 


Order 


Selection Criterion 

of 

Autoregressive 

Autoregressive 

Integrated 

Model 


plus constant 

autoregresi 



trend 

- 

1 

532.9 

464.3 

466.5 

2 

511.3 

484.3 

478.3 

3 

489.1 

431.9 

489.5 

4 

470.2 

474.3 

501.5 

5 

453.9 

464.9 

513.0 

6 

440.5 

456.0 

523.3 

7 

429.7 

447.9 

531.8 

8 

420.8 

440.6 

540.4 

9 

414.1 

434.8 

549.4 

10 

408.8 

429.3 

551 .5 

11 



552.8 

12 ’ 



549.9 

13 



546.4 

14 



538.7 

15 



532.4 


Table IV-14. 


Coefficients for Band 3 Selected Wheat Scene Model 


Coefficient 


Estimated Value 



.093225 

.099891 

.099302 

.10021 

.093787 

.099604 

.099389 

.10044 

.10047 

.099348 

.10040 ' 





t 

CO 

cn 

i 


Tabl e 

IV-1.5. Identified 

Models for Band 

3 of Combined Scene 

Order 

Selection Criterion 


of 

Autoregressive 

Autoregressive 

Integrated 

Model 


plus constant 
trend 

autoregressive 

1 

489.3 

436.8 

452.7 

2 

462.8 

447.0 

460.9 

3 

440.5 

440 . 6 

470.3 

4 

421.7 

431.0 

480.8 

5 

406.9 

420.9 

488.8 

6 

396.0 

412.7 

493.2 

7 

388.6 

405.9 

499.8 

8 

384.0 

401 .0 

505.1 

9 

381.4 

397.5 

504.6 

TO 

380.3 

394.7 

505.9 

1‘1 

380.2 

392.7 

505.9 

12 

380.6 

391.6 

504.6 

13 

381.5 

391 .0 

502.6 

14 

383.3 

391.6 

500.5 

IS 

386.3 

392.5 

497.3 


Table I.V-T6. Coefficients 

for Band 3 Selected Combined Scene Model 

Coef f i c lent 

Estimated Value 

a 1 

..099769 

4 

.098600 


.10137 

4 

.10010 

a 5 

.10057 

4 

.097140 

4 

.10123 

4 

.10080 

o 

a 9 

.097301 

a io 

.098323 

a 11 

.10038 
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selection criterion in Table IV-17. The model chosen on the basis of 
the selection criterion is the first order autoregressive plus constant 
trend term. The constant trend term here accounts for a mean valge in 
the driving noise which is not adequately modeled in the initial condi- 
tions taken from the empirical data. The residual variance for the. 
first order autoregressive plus constant trend model is 4.43 x 10 ^ thus 
indicating a good fit to the empirical data. The estimated coefficients 
for this model are given in Table IV-18. 

The three basic model types were also identified for band 4 of the 
combined scene. The identified models are listed according to order and 
selection criterion in Table IV-19. It is clear that on the basis of 
the selection criteria, the first order autoregressive plus constant 
trend model is to be chosen. The model is a good fit to the empirical 
data as is exhibited by the 1.20 x 10 ^ residual variance. The estimat- 
ed coefficients for this model are listed in Table IV-20. 

E) Band 5_ 

The wavelength intervals for band 5 are .8517 - .9910 pm for the 
wheat scene and .8481 - .9850 urn for the combined scene. The three 
basic model types were identified for the wheat scene. The identified 
models are listed according to order and selection criterion in Table 
IV-21. It is clear from the value of the selection criterion that the 
first order autoregressive model is to be chosen. It is also noted that 
the other two model types have selection criterion very close to the 
one chosen. Hence, if there is any difficulty in validation of the 
selected model, two alternative model types are available. It is in- 
teresting to note that all three model types have a first order model 
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Table IV-17. Identified Models for Band 4 of Wheat Scene 


Order 

Selection 

i Criterion 


of 

autoregressive 

autoregressive 

integ rated 

Model 


plus constant 
"trend 

autoregress 

-1 

469.4 

433.3 

433. 1 

2 

439.2 

471.3 

450.0 

3 

417.3 

454.3 

459,3 

4 

403.8 

442.1 

463.0 

5 

396.7 

433.6 

473.6 

6 

394.7 

423.7 

477.4 

7 

395.3 

425.4 

480.2 

8 

393.0 

42^.6 

481.1 

9 

400.2 

422.9 

481.8 

10 

4'03 .6 

423.4 

482.-0 

11 

406.9 

423.8 

480.8 

12 

410.2 

425.6 

477.-0 

<13 

411 .9 

425.8 

474.5 

14 

412.7 

426.7 

472.4 

15 

4T2:5 

426.1 

469.1 

16 

411.4 

426.5 

466.9 

17 

410.2 

-424.9 

464.1 

18 

408.3 

423.4 

461.2 

T9 

407.7 

421.5 

458-1 

20 

406.7 

4 T9. 4 

455.1 


Table I>V-13. Coefficients .for Band 4 Selected Wheat Scene 'Model 
Coefficient Estimated Value 




.93593 

.13969 
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Table IV-19. Identified Models for Bond 4 of Combined Scene 




Selection Criterion 


Order of 

Autoregressive 

Autoregressive 

Integ rated 

Model 


pi us' constant 
trend 

autoregressive 

1 

402.5 

418.3 

357.2 

2 

379.3 

403.6 

394.8 

3 

364.3 

390.9 

400.8 

4 

358.8 

382.9 

403.3 

5 

358.3 

379.1 

404.7 

6 

359.0 

376.9 

403.3 

7 

362.3 

376.8 

407.8 

8 

364.6 

376.7 

408.6 

9 

366.6 

376.4 

408.4 

10 

363.4 

376.4 

406.8 

11 

363.6 

376.6 

406.5 

12 

363.8 

375.7 

405.0 

13 

367.9 

374.8 

402.6 

14 

366.0 

372.4 

401.7 

15 

364.2 

371.2 

400.0 


Table IV-20. Coefficients for Band 4 Selected Combined Scene Model 


Coefficient Estimated Value 


a .93382 

c 1 .15933 
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Table 

IV— 21 - Identified 

Models for Band 5 of Wheat Scene 
Selection Criterion 

Order 

Model 

of 

Autoregressive 

Autoregressive 
pl-us constant 
trend 

Integrated 

autoregressive 

1 


339.9 

339.7 

738.4 

2 


•*74.1 

7 -33.6 

736.3 

3 


330.6 

770.0 

334.3 

4 


328.4 

327.7 

7 7 2.1 

5 


327.4 

327.7 

331.7 

6 


326.3 

325.2 

329.7 

7 


326.5 

325.7 

329.2 

3 


326.2 

725.0 

328.4 

9 


324.8 

323.9 

329.3 

10 


322.6 

321.7 

327.0 


Table IV-22. Coefficients for Band 5 Sel ected- Wheat Scene Model 


Coefficient 


Estimated Value 
1.00055 


PACT? to 

0P *X» 
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with the best selction criterion. This indicates that the empirical 

data in this band is fairly well behaved and that simple models of the 

spectral response suffice for purposes of representation . The residual 

-3 

variance of the first order autoregressive model is 2.80 x 10 , thus 

indicating a good fit to the empirical data. The estimated coefficient 

of the selected model is given in Table IV-22. 

For the combined scene, the three basic models were identified. 

The identified models are tabulated according to order and selection 

criterion in Table IV-23. According to the selection criterion, the 

third order autoregressive model is to be chosen. The residual variance 

-3 

for this model is 3.86 x 10 , an indication of a good fit to the empir- 

ical data. The estimated coefficients for the selected third order au- 
toregressive model are listed in Table IV-24. 

F) Band 

The spectral intervals included in band 6 are .9910 - 1.130 jim for 
the wheat scene and .9350 - 1.122 um for' the combined scene. The three 
basic models were identified for the wheat scene and are listed accord- 
ing to order and selction criterion in Table IV-25. It is seen from 
the table that the second order autoregressive plus constant trend model 
is to be chosen on the basis of the selection criterion. This model has 
a residual variance of 1.44 x 10 J which is an indication of a qood fit 
to the empirical data. The estimated coefficients for the selected 
model are given in Table IV-26. 

The model types identified for band 6 of the combined scene are the 
same three basic types identified for band 6 of the wheat scene. The 
identified models are tabulated according to order and selection cri- 
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Table IV-23. Identified Models for Band 5 of Combined Scene 


Selection Criterion 


Order of 
Model 

Autoregressive 

Autoregressive 
plus constant 
trend 

Integrated 

autoregress 

1 

307.2 

i 

307.3 

308.3 

2 

310.3 

309.7 

306.7 

3 

313.8 

313.0 

307.5 

4 

309.5 

308.7 

305.7 

5 

306.5 

' 305.9 

304.1 

6 

306.2 

305.5 

303.3 

7 

304.3 

304.0 

302.3 

■8 

302.3 

301.8 

300.5 

9 

301.0 

300.5 

299.9 

10 

299.0 

298.6 

298.4 


Table IV-24. Coefficients for Band 5 Selected Combined Scene Model 
Coefficients Estimated Value 


a 1 

a 2 

a-. 


.38814 

.28995 

.32304 
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Table IV-25. Identified Model’s for Band 6 of Wheat Scene 

Selection Criterion 


Order of 
Model 

Autoregress ive 

Autoregressive 
plus constant 
trend 

Integ rated 
autoregressive 

1 

374.4 

374.9 

370.0 

2 

375.8 

376.7 

368.9 

3 

370.5 

371.9 

366.2 

4 

369.3 

371.6 

369.1 

• 5 

364.0 

366.3 

367.7 

6 

359.0 

362.5 

367.1 

7 

355.4 

359.3 

365.3 

8 

351.4 

356.2 

367.4 

9 

348.2 

352.1 

365.2 

10 

343.5 

345.2 

364.2 


Table IV-26. Coefficients for Band 6 Selected Wheat Scene Model 
Coefficient Estimated Value 


.51004 

.47772 

.14773 
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tenon in Table IV-27. Based on the selection criterion, the first 

order autoregressive model is chosen. It has only a si ighty better 

selection criterion than the first order autoregressive plus constant 

trend model. Thus a competitive alternate model is available. The 

-3 

residual variance of the selected model is 1.91 x 10 thus indicating a 
good fit to the empirical data. The estimated coefficient for the 
selected model is given in Table IV-28. 

G) Band 7 _ 

Band 7 consists of the spectral intervals 1.130 - 1.344 pm for the 
wheat scene and 1.122 - 1.307 um for the combined scene. The three 
basic model types identified 'for the preceeding spectral bands’ were also 
identified for band 7 of the wheat scene. The identified models are 
listed according to order and selection criterion in Table IV-29. Ac- 
cording to the selection criterion, the eighth order integrated autore- 
gressive model is to be chosen. However, the model with the best selec- 

I 

tion criterion that also passes all of the validation tests is the fifth 
order integrated autoregressive model. The residual variance of the 
selected fifth order integrated autoregressive model is 2.42 x 10~^ 
which indicates a good fit to the empirical data for band 7 of the wheat 
scene. The estimated coefficients for the selected model are given in 
Table IV-30. 

The three basic models were identified for band 7 of the combined 
scene. The identified models for the combined scene are given according 
to order and selection criterion in Table IV-31 . According to the 
selection criterion, the fifteenth order integrated autoregressive model 
is to be chosen. However, the fifteenth order integrated autoregressive 
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Table IV-27. Identified Model.s for Band 6 of Combined Scene 


Selection Criterion 


Order of 

Autoregressive 

Autoregressive 

Integrated 

Model 


pi us constant 
trend 

autoregressive 

1 . 

355.9 

355.8 

353.9 

2 

347.8 

346.7 

353.1 

3 

338.5 

'337.5 

351.8 

4 

332.2 

332.5 

351.3 

5 , 

329.1 

328.5 

349.2 

6 

326.2 

327.7 

348.7 

7 

325.7 

328.2 

347.8 

8 

352.6 

326.4 

347.5 

9 

323.7 

325.4 

345.4 

10 

322.5 

323.4 

345.4 


Table IV-28. Coefficients for Band 6 Selected Combined Scene Model 
-Coefficients Estimated Value 

a ‘ 1.00091 
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Table IV-29. Identified Models for Band 7 of Wheat Scene 


Selection Criterion 


Order of 
Model 

Autoregressive 

Autoreg ressive 
plus constant 
trend 

Integrated 

autoregressive 

1 

456.6 

454.3 

457.3 

2 

437.3 

435.6 

457.4 

3. 

429.0 

427.0 

460.2 

4* 

425.0 

424.8 

461.6 

5 

425.5 

424.2 

464.8 

6 

425.9 

424.2 

462.5 

7 

425.2 

423.3 

464.1 

8 

424.8 

423.1 

469.3 

9 

427.1 

427.5 

469.1 

TO 

429.5 

427.2 

467.2 


Table IV-30. Coefficients, for Band 7 Selected Wheat Sc'ene Model 


Coefficient 


Estimated Value 


a 


1 


a 

a 

a 

a 


2 

3 

4 

5 


. 1'4802 

.06755,9 

.70349 

.11204 

.10533- 



Table IV-31 . Identified Models for Band 7 of Combined Scene 


Selection Criterion 


Order of 
Model 

Autoregressive 

Autoregressive 
plus constant 
trend 

Integrated 

autoregressive 

1 

231.2 

231.4 

231.5 

2 

229.7 

229.8 

230.3 

3 

228.4 

228.3 

229.0 

4 

227.1 

227.0 

227.8 

5 

225.8 

225.7 

227.6 

, 6 

225.6 

225.9 

226,. 1 

7 

224.1 

261.5 

259.2 

8 

257.5 

265.0 

260.2 

9 

259.1 

267.0 

259.9 

10 

259.4 

265.5 

258.8 

11 

258.2 

265.0 

258.9 

12 

258.4 

264.7 

260.7 

13 

259.5 

263.4 

259.2 

14 

258.0 

261.8 

258.7 

15 

257.3 

261.4 

253.8 


Table IV-32. Coefficients for Band 7 Selected Combined Scene Model 


Coefficients 


Estimated Value 



c 


.64063 

.17721 

.11417 

.038031 

.10856 

-.058380 

-.038789 

-.061748 

-.017016 

.49218 
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model fails the correlogram validation test. Hence, the next alternate 
model is chosen according to Table IV-31 . This is the ninth order au- 
toregressive plus constant trend model. This model passes the val ida- 

_2 

tion tests and the residual variance is 3.27 x 10 . Hence the fit to 

the empirical data is good. The estimated coefficients for this model 
are given in Table IV-32. 

H) B3nd 8 

The spectral intervals included in band 8 are 1.446 - 1.821 pm for 
the wheat scene and 1.451 - 1.818 pm for the combined scene. The three 
basic models identified for the other spectral bands are also identified 
for band 8. The identified models for this spectral band of the wheat 
scene are tabulated according to order and selection criterion in Table 
IV-33. On the basis of the selection criterion, the ninth order in- 
tegerated autoregressive model is to be selected. Unfortunately, this 
model does not pass the cumulative periodogram validation test. The 
model selected as an alternative is the eighth order integrated autore- 
gressive model. This model passes all the validation tests. The resi- 
dual variance for the selected model is 3.18 x 10 ^ which indicates a 
good fit to the empirical data for this band of the wheat scene. Table 
IV-34 gives the estimated coefficients for the selected eighth order in- 
tegraated autoregressive model . 

The three basic models were also identified for band 8 of the com- 
bined scene and are listed according to order and selection criterion 
in Table ,IV-35. The eighth order integrated autoregressive model is 
chosen on the basis of the selection criterion. The residual variance 
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Table 

IV-33. Identified 

* 

Models for Band 

8 of Wheat Scene 



Selection Criterion 


Order 

of 

Autoregressive 

Autoregressive 

Integrated 

Model 



plus constant 
trend 

autoregressive 

1 


469.9 

494.7 

495.7 

2 


447.8 

457.8 

506.8 

3 


427.8 

436.0 

519.4 

4 


422.1 

429.9 

526.4 

5 


426.1 

434.2 

530.2 

6 


433.1 

441.3 

540.5 

7 


441.9 

455.2 

538.5 

8 


448.1 

463.1 

555.8 

9 


453.8 

470.7 

556.9 

10 


457.7 

475.9 

554.5 

11 


460.0 

477.5 

548.3 

12 


457.3 

477.5 

547.2 

13 


455.5 

476.2 

544.4 

14 


454.8 

477.3 

546.2 

15 


457.4 

480.3 

547.2 


Table IV-34. Coefficients for Band 8 Selected Combined Scene Model 


Coefficients 


Estimated Value 


a 

a 

a 


1 

2 

3 


a 

a 

a 

a 

a 


4 

5 

6 

7 

8 


.11977 

.10839 

.10895 

.10525 

.10025 

.11203 

.10458 

.093896 



Table IV-35. Identified Models for Sand 3 of Combined Scene 




Selection Criterion 

Order of 

Autoregressive 

Autoregressive 

Integrated 

Model 


plus const-ant 
trend 

autoregressive 

1 

275.6 

279.6 

277.2 

2 

271.9 

274.6 

276.5 

3 

' 270'. 7 

273.9 

275.8 

4 

270.2 

•273.2 

274.5 

5 

268.9 

272.3 

273.0 

6 

267.4 

271.3 

272.7 

7 

266.6 

270.4 

271.4 

8 

265.0 

308.7 

319.7 

9 

310.2 

303.3 

318.8 

10 

310.3 

303.7 

319.4 

11 

309.7 

309.5 

319.4 

12 

310.0 

309.6 

31 8., 4 

13 

309.7 

308.3 

316.7 

14 

308.4 

306.7 

315.1 

15 

306.9 

305.3 

313.4 


Table IV-36. Coefficients for Band S Selcted Combined Scene Model 


Coefficients Estimated Value 



.074048 

.054155 

.10216 

.058484 

.080674 

.090834 

.083258 

.11358 
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of the selected model is 9.53 x 10 "* thus indicating a good fit to the 
empirical data for band S of the combined scene. Table IV-36 gives the 
estimated coefficients for the selected model. 

_I) Band 9^ 

The spectral intervals for this band are 1.959 - 2. 336 ym for the 
wheat scene and 1.967 - 2.356 ym for the combined scene. The three 
basic model types were identified for this band of the wheat scene and 
are tabulated according to order and selection criterion in Table IV-37. 
According to the selection criterion, the sixth order integrated au- 
toregressive model is selected. The residual variance for the selected 
model is 8.00 x 10 ^ thus indicating a good fit to the empirical data 
for band 9 of the wheat scene. Table IV-38 gives the estimated coeffi- 
cients for the se.lected model. 

The three basic models were also identified for band 9 of the com- 
bined scene. Table IV-39 lists the identified models according to order 
and selction criterion. According to the selection criterion, the first 
order integrated autoregressive model should be chosen. However, this 
model is judged not to pass the qualitative periodogram validation 
test. Therefore, the alternative selected model is the first order au- 
toregressive model. This model passes the validation tests. The resi- 
-2 

dual variance for the first order autoregressive model is 1.18 x 10 
thus indicating a good fit to the empirical data for band 9 of the com- 
bined scene. Table IV-40 gives the estimated coefficient for the 
selected first order autoregressive model. 

We now have selected models for all the defined bands of both the 
wheat scene and the combined scene. Next we validate the selected 



Table IV-37. Identified Models for Band 9 of Wheat Scene 


Selection Criterion 


Order of 
Mode! 

Autoregressive 

Autoregressive 
plus constant 
trend 

Integrated 

autoregressive 

1 

546.3 

535.9 

551 .2 

2 

516.3 

518.5 

559.0 

3 

504.4 

501.5 

565.3 

4 

495.7 

494.9 

569.4 

5 

495.9 

495.3 

572.6 

6 

501.0 

499.6 

575.2 

7 

508.5 

507.4 

573.9 

8 

514.8 

514.4 

572.1 

9 

517.8 

51'8.3 

571.6 

10 

519.4 

520.4 

570.1 


Table IV-38'. Coefficient's foir Band 9 Selected Wheat Scene Model 


Coefficient 


Estimated Value 


a 1 

a 2 

a 3 

a 4 

a 5 

a 6 


.10952 

.096485 

.10262 

.11044 

.11506 

.11766 
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Table IV-39. Identified Models for Band 9 of Combined Scene 




Selection Criterion 


Order of 

Autoregressive 

Autoreg ressive 

Integrated 

Model 

\ 

plus constant 
trend 

autoregressive 

1 

356.1 

355.3 

356.5 

2 

348.7 

348.3 

355.1 

3 

346.5 

346.5 

354.0 

4 

345.4 

345.6 

353.0 

5 

344.0 , 

344.0 

351.5 

6 

342.3 

342.5 

349.9 

7 

341.1 

341.2 

352.5 

8 

342.1 

341.3 

351 .2 

9 

341.4 

340.5 

350.1 

10 

340.3 

339.0 

348.5 


Table IV-40. Coefficients for Band 9 Selected Combined Scene Model 
Coefficient Estimated Value 


99759 
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models. Finally, these models are used for average information calcula- 
tions in Chapter V. 

4_. Val idation of Identified Model s . 

In this section val idation of the identified models for each band 
of both scene types is carried out. The val idaion techniques used here 
are those developed in Chapter III. We consider th'e validation of the 
selected models by bands as in the previous section of this chapter. 

The validation tests are implemented wih computer programs written in 
FORTRAN and included in Appendix III. 

A) Band 1_ 

We first consider val idation of the selected sixth order autore- 
gressive model for band 1 of' the wheat scene. 

1) Zero Mean Test 

This test will be carried out for all bands of both scene types at 
a significance level of .01 so that there is a standard basis of compar- 
ison for all selected models. The value of ng at this level of signifi- 
cance for the number of samples in this research (approximately 115 to 
160 samples) is ng = 2.62 CAST. 

The value of the test statistic given by “the selected model for the 
wheat scene is 

| t ( x) | = 1.431 X'10" 1 (4.2) 

Hence, the selected sixth order autoregressive model easily passes^this 


test. 
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2) Serial Independence Test 

This test is conducted on all bands of both scene types at a signi- 
ficance level of .01 so that there is a standard basis of comparison 
for all selected models. The critical value,, ng, is dependent on the 
value of nl used for this test. In this study nl = .IN (N is the number 
of empirical data points) for all models tested. The critical values 
are listed for several values of nl in Table IV-4-1 CA53. 

The test statistic for the selected sixth order autoregressive 
model for band 1 of the wheat scene is 

n(x) = 1.515 x 10 1 (4-3) 

for nl = 10. It is clear from Table IV-41 that the selected model 
passes the serial independence test'. 


Table IV-41. Cri.tical Values for Serial Independence Test 


nl 

Critical Value, 

9 

21.6660 

10 

23.2093 

11 

24.7250 

12 

26.2170 

13 

27.6883 

14 

29.1413 


15 


30.5779 
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3) Cumulative Periodogram Test 

This test is carried out on all bands of both scene types at a pro- 
bability level of .99, thus giving a standard basis of comparison for 
all selected models. In all cumulative periodogram plots, the boun- 
daries that determine the success of the test are, of course, the two 
parallel l ines. 

The selected model for band 1 of the wheat scene passes this test 
as is seen from Figure IV-3. 

4) Correlogram Test 

It is seen from figure IV-4 that the selected sixth order autore- 
gressive model passes this test. 

In the correlogram plots, the test boundaries are shown as dashed 
l ines. 

5) Periodogram Test 

As seen in Figure I.V-5, the selected model for band 1 of the wheat 
scene may be judged to pass this qualitative test. 

In the periodogram plots, the periodogram for the empirical data is 
plotted as a solid line and the periodogram for the synthetic data gen- 
erated from the candidate model is plotted as a dashed line. For most 
cases, the two plots are so nearly coincident as to be indistinguish- 
able. 

Next we consider the validation of the selected eleventh order in- 
tegrated autoregressive model of the second kind for band 1 of the com- 
bined scene. 


6) Zero Mean Test 
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, The test statistic for the selected model is 

|t(x) | = 2,184'x 10 _1 . -(-4-^4 7 

Wence, -the' sel ected model easi ly passes th-is test. 

7) '-Serial Independence Test 

As noted previously, several of the identified models with higher 
selection criterion do not pass this test. As an- example, the 
fifteenth order autoregressive model gives the test statistic 

n (x) = 7^.434 x 10 1 '(4-5) 

for nl = 9. Thus the model clearly fails 'the test. Other. models with 
higher selection criterion similarly failed thistest. The selected 
model has the test statistic 

n(x) = 2.009 x 10 1 (4-6) 

for nl = 10. Thus, the el even-th order integrated autoregressive model 
of the second kind passes this test. 

8) Other Tests 

Figures IV-6, IV-7, ad TV-S show that the -sel ected -model for band 1 
of the combined- scene passes the cumulative penodogram, correlogram, 

‘and penodogram tests. 

Hence, we have va I idated -model s for band -1 of both scene- types. 
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B) . Band 2_ 

The validation of the selected second order autoregressive model 
for this band of the wheat scene is considered first. 

1) Zero Mean Test 

The test statistic for the selected model is 

| t ( x) | = 1.955 (4-6) 

Thus the selected model passes this test. 

2) Serial Independance Test 

The test statistic for the second order autoregressive model is 

n (x> = 1.869 x 10 1 (4-7) 

for nl = 11. Hence, the selected model passes the serial independance 
test. 

3) Other Tests 

It is seen from Figures IV-9, IV-10, and IV— 1 1 that the selected 
model for band 2 of the wheat scene passes the cumulative periodogram, 
correlogram, and periodogram tests. 

Next, we discuss the validation of the selected second order au- 
toregressive model for band 2 of the combined scene. 

4) Zero Mean Test 

The test statistic for the selected model is 
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9 

1 1 ( x) | = 1.055 C4-8) 

Hence, the selected model easily passes this test. 

5) Serial Independance Test 

The selected model gives the test statistic 

n<x) = 1.365 x 10 1 (4-9) 

for nl = 11. Hence, the selected model also passes this test. 

6) Other Tests 

The selected model for band 2 of the combined scene passes the cu- 
mulative periodogram, correl ogram, and periodogram tests as is seen from 
Figures IV-12, IV-13, and IV-14. 

C) Band 3_ 

Validation of the selected eleventh order integrated autoregres- 
sive model for band 3 of the wheat scene is considered first. 

1) Zero Mean Test 

The test statistics for the zero mean test is 

i t (x) | = 3.616 x 10" 1 . (4-10) 

Hence, the 'selected model easily passes this test. 

2) Serial Independence Test 

The selected model gives the test statistic 
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nCx) = 1.153 x 10 1 . <4-1 1 ) 

for nl - 10. Thus, the selected model easily passes this test. 

3) Other Tests 

Figures IV-15, IV-16, and IV-17 show that the selected model for 
band 3 of the wheat scene passes the cumulative periodogram, correlo- 
grasn, and periodogram tests. 

The selected eleventh order integrated autoregressive model for 
band 3 of the combined scene is considered next. 

4) Zero Mean Test 

The test statistic for the selected model is 

[ t ( x) | = 1.871 x 10 -1 . (4-12) 

Thus, the selected model ^easily passes this test. 

5) Serial Independance Test 

The selected model gives the test statistic 

n(x) = 2.125 x 10 1 (4-13) 

for nl = 10. Thus, the selected model- passes this test. 

6) Other Tests 

It is seen from Figures IV-18, IV-19, and IV-20 that the selected 
model for band 3 of the combined scene passes the cumulative periodo- 
gram, correl ogram, and periodogram tests. 

Hence, we have validated models for band 3 of both scene types. 
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D_) Pond 4_ 

Validation of the selected first order autoregressive plus constant 
trend model for band 4 of the wheat scen a is considered first. 

1) Zero Mean Test 

Tne test statistic for the selected cede! is 

|t(x)| = 1.753 x Iff 1 . (4-14) 

Hence, the selected model easily passes this test. 

2) Serial Independence Test 

The selected model gives the test statistic 

n(x> = 2.00 x 10 1 (4-15) 

fo nl = 12. Therefore, the selected model also passes tfns test. 

3) Other Tests 

Figures IV— 21 , IV- 22, and IV-23 show that the selected model passes 
the cumulative penodogram, correl ogram, and penodogram tests. 

We next consider validation of the selected first order autore- 
gressive plus constant trend model for band 4 of the combined scene.' 

4) Zero Mean Test 

The test statistic for the selected model is 

It(x) [ = 1.593 x 10" 1 . (4-16) 

Hence, the selected model easily passes this test. 


5) Serial Independence Test 
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The selected model gives the test statistic 

/ 

rj(x) = 2.1250 x 10 1 • (4-17) 

for nl = 12. Hence, the selected model also passes this test. 

6) Other Tests 

It is seen from' Figures IV-24, IV-25, and IV-26 that the selected 
model passes the cumulative periodogram, correlogram, and periodogram 
tests. 

Hence, we have validated models for both scene types of band 4. 

E) Band 5_ 

First, validation of the selected first order autoregressive model 
for band 5 of the wheat scene is considered. 

1) Zero Mean Test 

The test statistic for the selected model is 

|t(x) | = 2.495 x 10” 1 . (4-18) 

Hence, the selected model easily passes this test. 

2) Serial Correlation Test 

The selected model yields the test statistic 

n(x) = 1.737 x 10 1 (4-19) 

for nl =11. Therefore, the selected model pases this test. 


3) Other Tests 
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Figures IV-27, IV-28, and IV-29 show that the selected model passes 
the cumulative periodogram, correlogram, and periodogram tests. 

Next, vaidation of the selected third order autoregressive model 
for band 5 of the combined scene is considered. 

4) Zero Mean Test 

The test statistic for the selected model is 

|t(x> | = 3.622 x 10~ 2 . (4-20) 

Thus, the selectd model easily passes this test. 

5) Serial Independence Test 

The selected model gives the test statistic 

n(x) = 1.628 x 10 1 (4-21) 

for nl =11. Hence, the selected model passes this test. 

6) Other Tests 

Figures IV-30, IV— 31 , and IV-32 show that the selected model passes 
the cumulative periodogram, correlogram, and periodogram tests. 

Thus we have val idated model s for band 5 of both scene types. 

F) Band 6 

Validation of the selected second order autoregressive plus con- 
stant trend model for band 6 of the wheat scene is considered first. 


1) Zero Mean Test 






“ 133 “ 


The test statistic for the selected model is 

|t(x)| = 1.299 x 10 _1 . (4-22) 

T^hus, the selected model easly passes this test. 

2) Serial Independence Test 

The selected model gives the test statistic 

n<x) = 2.345 x 10 1 (4-23) 

for nl = 11. Hence, the selected model passes this test. 

3) Other Tests 

The selected model passes the cumulative periodogram, correlogram, 
and periodogram tests as is seen from Figures IV-33, IV-34, and IV-35. 

Next, validation of the selected first order autoregressive model 
for band 6 of the combined scene is considered. 

5) Zero Mean Test 

The test statistic for the selected model is 

1 1 ( x) | = 2.861 x 10" 1 (4-24) 

Hence, the selected model easly passes this test. 

5) Serial Independence Test 

The selected model yields the test statistic 

n (x) = 8.322 (4-25) 

for nl =-11. Hence, the selected model also easily passes this test. 





135 - 


6) Other Tests 

It is seen from Figures IV-36, IV-37, and IV-38 that the selected 
model passes the cumulative per iodogram, - correl ogram, and periodogram 
tests. > 

Thus, we have validated models for band 6 of both scene types. 

G) Band 7 _ 

Validation of the selected fifth order integrated autoregressive 
model for band 7 of the wheat scene. 

1) Zero Mean Test 

The selected model gives the test statistic 

| t ( x) | = 1.853 (4-26) 

Hence, the selected model easily passes this test. 

2) Serial Independence Test 

The test statistic for the selected model is 

n(x) = 1.807 x 10 1 (4-27) 

for nl = 14. Hence, this test is passed by the selected model. 

3) j 0ther Tests 

figures IV-39, IV-40, and IV-41 show that the selected model passes 
the cumulative periodogram, correlogram, and periodogram tests. 

We next consider validation of the selected ninth order autoregres- 
sive plus constant trend model. 




Fiv. IV-36* Cumulative Periodogram 
Band 6 , Combined Scene 



I 
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4) Zero Mean Test 

The selected model gives the test statistic 

| t(x) | = 1.-208 x 10" 1 (4-28) 

Thus, the' selected model easily passes this test. 

5) Serial Independence Test 

The test statistic for the selected model is 

n<x) = 5.568 (4-29) 

for nl = 15. Therefore, the selected mddel easily passes this test. 

6) Other Tests 

Figures IV-4-2, IV-43, and IV-44 show that the selected model passes 
the cumulative periodogram, correlogram, and periodogram tests. 

Hence, the selected models are validated for band 7 of both scene 
types. 

H) Band 8^ 

Validation of the selected eighth order integrated autor'egressive 
model for band 8 of the wheat scene is considered first. 

10 Zero Mean Test 

The test statistic for the selected model is 

1 1 ( x) | = 7.217 x 1‘0 -1 (4-30) 


Thus, the selected model passes this test. 
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2) Serial Independence Test 

The selected model gives the test statistics 

n(x) = 2.516 x 10 1 <4-31) 

for nl = 13. Hence, the selected model passes the test. 

3) Other Tests 

The selected model passes the cumulative periodogram, correlogram, 
and periodogram tests as is seen from Figures IV-45, IV-46, and IV-47. 

Next, validation of the selected eighth order integrated autore- 
gressive model or band 8 of the combined scene is considered. 

4) Zero Mean Test 

The selected model gives the test statistic 

|t(x) | = 1.033 (4-32) 

Thus, the selected model easily passes this test. 

5) Serial Independence Test 

The test statistic for the selected model is 

n(x) = 3.141 (4-33) 

for nl = 13. Therefore, the selected model passes this test. 

6) Other Tests 

Figures IV-48, IV-49, and IV-50 show that the selected model passes 
the cumulative periodogram, correlogram, and periodogram tests. 





142 - 


Hence, we have validated models for band 3 of both scene types. 

Jp Band <?-- - 

Validation of the selected sixth order integrated autoregressive 
model for band 9 of the wheat scene is considered first. 

1) Zero Mean Test 

The selected model gives the test statistic 

1 1 ( x) | = 8.142 x 1.0" 1 (4-34) 

Thus, the selected model easily passes this test. 

2) Serial Independence Test 

The test statistic for the selected model is 

n(x> = 9.915 (4-35) 

for nl = 15. Therefore, this test is passed by the selected model. 

3) Other Tests 

The selected model passes the cumulative periodogram, correlogram, 
and periodogram tests as is 'seen in Figures IV-51, IV-52, and IV-53. 

Next, validation of the selected first order autoregressive model 
for band 9 of the combined scene is considered. 

4) Zero Mean Test 

The test statistic for the selected model is 
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| tCx) | = 9.324 x 10" 1 (4-36) 

Thus, the selected model easily passes this test. 

5) Serial Independence Test 

The selected model gives the test statistic 

n (x) = 1.362 x 10 1 (4-37) 

for nl = 15. Therefore, the selected model passes this test. 

6) Other Tests 

Figures IV-54, IV-55, and IV-56 show that the selected model passes 
the cumulative periodogr.am, cornel ogram, and periodogram validation 
tests. 

Hence, we have validated models for band 9 of both scene types. 

5_. Cone l usion 

Models of nine spectral bands for two empirical data sets have been 
identified, selected and validated. The validated models of the two 
scene types are given in Table IV-42 for easy reference. 
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Table IV-42. Validated Models 


Band 

Wheat Scene 

Combined Sc 

1 

AR(6) 

IAR 2 (11) 

2 

AR(2> 

AR(2) 

3 

lARdl) 

IAR(II) 

4 

ARCC1 > 

ARC(I) 

5 

AR(1) 

AR<3) 

6 

ARC(2) 

AR (1 ) 

7 

IAR (5) 

ARCC9) 

8 

IARC8) 

IAR(8) 

9 

IARC6) 

AR(1 ) 



- 147 - 


Chapter V 

An Application of Information Theoretic Techniques 
_1_. Introduction 

This chapter demonstrates an application of the information 
theoretic techniques developed in Chapter II to studying some parameters 
of mul tispectral scanner systems. In particular, the techniques are ap- 
plied to the models constructed in Chapter IV for the two spectral scene 
types under consideration in this research. The average information 
criterion is used to select a subset of spectral bands. An attempt at 
estimation of classification accuracy for the hypothetical mul tispectral 
scanner is discussed. 

2 . Average Information Studies 

The average information computation techniques developed in Chapter 
II are used to study average information in the received spectral pro- 
cess about the spectral response process of the scene under observation. 
Recall that we are representing the spectral process received by the 
mul tispectral scanner by 
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y(k) = S(k) + n(k) , keCX^,)^ (5-1) 

where 

SOk) is the spectral response process of the scene 

and 

n(k) is the disturbance or noise process. 

The models constructed in Chapter IV are used for representing the spec- 
tral response process S(k) in each spectral band. As discussed in 
Chapter II, n(k) is assumed to be white noise with possibly different 
power spectral densi-ty Levels in different spectral bands. 

The first computation is average information in y(k) about S(k) as 
a funct.ion of spectral bandwidth for each spectral band of both scene 
types. The average information is computed for several values of the 
variance of the noise disturbance', o n . Since the noise disturbance is 

assumed' to be of constant power spectral density level for each spectral 

. ' 2 

band, considering several vaLues of o n has the effect of allowing the 
study of average information for sev.eral signal-to-noise ratio (SNR) 
conditions. 'Thus, the objective of studying the effects of spectral 
bandwidth and signaL-to-noise ratios as parameters of multispe.ctral 
scanners is achieved in these computations. The average information 
computations are made with the use of a computer program written in FOR- 
TRAN. A copy of the computer program is included in Appendix III for 
reference. The results of these computations are displayed graphically 

in Figures V-1 to V-18. It is noted that these figures have curves 
2 

plotted with as a running parameter. Also, the curves are plotted as 
a function of the number of points in the spectral interval . *Th-is has 
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the advantage of making the curves applicable to the same models for 
different spectral .intervals. 

Table V-1 gives the total average information for the defined spec- 
tral bands of the wheat scene foi several values of the noise variance 
2 

o^. Table V~2 gives simitar data for the combined scene. When consid- 
ering the results in Tables V-T and V-2, it must be remembered that the 
spectral bands are of different spectral bandwidths. Hence, the average 
information in spectral bands of approximately the same spectral 
bandwidth may be more useful in selecting subsets of bands. 

Thus the technique considered in this research is used to compute 
average information using the spectral models constructed for the de- 
fined spectral' bands. 

3_. Sel ection of £ Subset of Spectral Sands 

We now demonstrate a simple application of using average informa- 
tion to select a subset of spectral bands for inclusion on a multispec- 
tral scanner. For the purpose of this demonstration we make the follow- 
ing assumptions.- First assume that a subset of six of the defined spec- 
tral bands is derived. This is not an unusual number of spectral bands 
to be used in an application (i.e. scene classification). The second 
assumption concerns the amount of observation noise to include in each 

spectral band. For the purposes of this simple example, we assume that 

2 -3 

the variance of the observation noise is o =10 for all the defined 

n 

spectral bands. This is clearly not a realistic assumption, but is suf- 
ficient for our simple demonstration. Third, it is assumed we are in- 
terested in ordering the preference of spectral bands on the basis of 
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-9. Average Information, Band 5, Wheat Scene 




Fig. V— 13. Average Information, Band J , Wheat Scene 



Fig. V-lA. Average Information, Band 7, Combined Scene 
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the average information each band would have in an equal spectral 

bandwidth. This tends to amel iorate the effect of wider spectral bands 
having more average information due only to their larger spectral 

bandwidth. It is thought that this method of comparison will tend to 1 
select the subset of spectral bands with the highest amount of average 
information with each band competing on a more equal basis. Based on 
these assumptions, the spectral bands are ranked in order of preference 
in Table V-3. 
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Table V-1 . Average Information for Wheat Scene Bands 


Noise Variance, a c 



10~ 5 

o 

I 

10~ 3 

-2 

10 

io' 1 

1 

57.07 

53.49 

34.50 

11.43 

3.95 

2 

50.05 

28.09 

10.52 

4.45 

2.75 

3 

51.02 

34.59 

20.35 

12.65 

8.28 

4 

61 .64 

52.92 

30.00 

11.69 

4.15 

5 

57.30 

55.58 

44.96 

23.55 

9.31 

6 

57.11 

53.90 

37.20 

14.81 

5.12 

7 

77.19 

74.63 

60.31 

34.59 

1'6.6 

8 

67.56 

56.77 

34.80 

18.96 

10.48 

9 

80.04 

73.23 

50.10 

26.53 

13.10 


Note: The information values are given in nats here. 



Table V-2. 

Average 

Information for 

Combined 

Scene Bands 

Band 

io " 5 

Noise 

10~ 4 

2 

Variance, o n 
io' 3 ' 

io “ 2 

io ' 1 

1 

56.19 

53.70 

41 .33 

23.09 

12.23 

2 

53.51 

38.54 

16.17 

6.10 

3.05 

3 

52.73 

39.10 

22.93 

13.73 

8.72 

4 

61.49 

57.54 

40.03 

17.71 

6.24 

5 

56.36 

55.11 

45.73 

21.44 

7.09 

6 

56.21 

53.81 

40.96 

20.05 

7.83 

7 

80.48 

80.26 

73.25 

63.93 

30.54 

8 

69.93 

69.31 

64.15 

44.38 

22.46 

9 

7-9.93 

79.33 

74.19 

51.72 

23.23 
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Table 


Rank 

1 

2 

3 

4 

5 

6 

7 

8 
9 


V-3. Order of Preference of 
and Combined Scenes 
Wheat Scene Band 

5 

7 

6 
9 
1 

8 
4 
3 
2 


Spectral Bands for the Wheat 

Combined Scene Band 

7 
9 

8 

5 
1 

6 
4 
3 

2 


It is noted in Table V-3 that although the ordering is different, 
the six highest ranking bands are the same for both the wheat scene and 
the combined scene. Band'1 is in the visible region of the spectrum tor 
both scene types <see Tables IV-3 and IV-4). The other five preferred 
bands are all in the infrared portion of the spectrum. Thus relative to 
our averge information criterion, the infrared portion of the spectrum 
is generally preferred to the visible portion of the spectrum since 
bands 2 and 3 are ranked lowest for both the wheat scene and the com- 
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bined scene. This tends to indicate that -future mul tispectral scanners 
systems should hay.e more spectral bands in the infrared portion of the 
spectrum. Indeed this is the case* with the’ thematic mapper to be placed 
on LA'NDSAT-D (see, for example, reference CL20). 

Of’ course, if there were different levels of noise disturbance in 
different spectral bands, the order of preference could be entirely dif- 
ferent. This research does, however, provide a systematic method for 
de'al mg with such circumstances. A more realistic application of this 
technique would require such an approach. 

One of the major uses of data obtained with mul tispectral’ scanner 
systems is cl ossification' of the observed scenes. Thus it is felt’that 
estimation of classification performance gives an important measure of 
the usefulness of a proposed subset of spectral bands. The estimation 
of classification error is an important and complicated topic in itself. 
Whitsitt and Landgrebe CW1] have recently spent considerable effort on 
this topic. A technique, used in this research, to estimate classifica- 
tion performance was developed by Lissack and Fu CL33. This technique 
assumes a Bayesian classification technique and provides a computational 
technique for estimating classification performance. An attempt was 
made to use the Lissack-Fu technique to estimate the classification per- 
formance of the six selected spectral bands for the two scene types stu- 
died in this research. The empirical data, however, had the unfor- 
tunate property of producing covariance matrices that were singular to 
the numerical accuracy of the (IBM 370) computer system used in the 
research.- Hence, a meaningful estimate of the classification perfor- 
mance was not possible with the present data set. Therefore, the clas- 
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sification performance characteristics of the selected spectral bands 
are left for future investigation. 

!L' Concl usions 

This chapter has demonstrated the application of information 
theoretic techniques for the study of some parameters of mul tispectral 
scanner systems. First, average information in a received spectral band 
was calculated for several power spectral density levels of observation 
noise. This computation allowed the study of such parameters as spec- 
tral bandwidth and signal -to-noise effects on average information. 
Secondly, a simple demonstration of the use of average information as a 
technique for selection of a subset of spectral bands was given. Final- 
ly, an attempt at studying the classification performance of the select- 
ed subset of spectral bands, indicated that a more detailed investigation 
of the problem was necessary. This was left for future investigation. 
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Chapter 6 


Conclusions 


1_. Discussion 

This thesis is devoted to development of techniques for analysis of 
some parameters of mul tispectral scanner systems. These techniques 
represent an initial effort to provide an analytical framework for what 
heretofore has been approached mainly in an empirical and ad hoc manner. 
The information theoretic techniques developed in Chapter II are suffi- 
ciently general that they can be used to explore many different practi- 
cal questions in the study of parameters of mul tispectral scanner sys- 

I 

terns for remote sensing. The modeling techniques developed in Chapter 
III are applicable to almost any scene type of interest in remote sens- 
ing. Furthermore, models developed in such a manner could, of course, 
be used for other research on spectral scenes. Chapters IV and V are an 
extended study on empirical data using the techniques developed in 
Chapters II and III. Chapter IV demonstrates the advantages of a sys- 
tematic approach to model, construction by examination of several hy- 
pothesized models. Thus several alternative models are constructed for 
each spectral scene. Chapter V demonstrates that, for the empirical 
data studied, the infrared portion of the spectrum deserves increased 
attention in mul tispectral scanner system design. 
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2 . Further Research 

There are several aspects of this research that merit further ex- 
ploration. Some of the more obvious topics are mentioned. 

(1) It may be of considerable practical interest to extend the in- 
formation theoretic results in Chapter II to the case of nonwhite obser- 
vation noise. Though more complicated, an expression for average infor- 
mation can again be related to the optimum Wiener-Hopf filter impulse 
response CHID. The- state variable formulation of this problem would 
prove to be very useful for handling observation noise that could be 
described by a dynamic model. An application of this extension might be 
to study the effects of an extraneous spectral signal such as produced 
by bare soil surrounding a vegetation scene of actual interest. 

(2) Another extension of this reseach might be to consider other 
models for spectral scenes. In particular, it may be fruitful to con- 
sider moving average models or combined autoregressive-moving average 
models CB1DI Such an extension may result in lower order models for 
spectral scenes. However, identification of such models is more compli- 
cated than the cases considered in this thesis LK3D. 

(3) Extension of the scalar models to the vector model case might 
be interesting. This could have an application in temporal studies of 
spectral scenes. That is, models of the spectral response of vegetation 
scenes and their change over the growing season would be very useful 
when considering mul tispectral scanner system design. 

(4) An important ex-tension of this research is the consideration 
of the relationship between the information theoretic methods for 
selecting a subset of spectral bands and the accuracy of scene ciassifi- 
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cation using these bands. A simple experiment of this nature was at- 
tempted in Chapter V and met with d i-f f icut ties in estimation of some re- 
quired covariance matrices. Nevertheless, it would be extremely useful 
to study the efficacy of the mfcimation theoretic bond selction appli- 
cation in relation to the classification problem. It is expected that 
such things as types of models used for the spectral scenes and, indeed, 
the particular spectral scenes considered, would cause this to be a wide 
ranging and complicated study. Different classification tecniques might 
be expected to produce widely differing results when using a set cf 
bands selected by the information theoretic criterion. Indeed, proper 
consideration of the many .variabl es in such a study has been CW13 and 
should continue to be an area of fruitful research. 
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Appendix I 

Kalman Filter Algorithm 

The Kalman filter algorithm is included as a reference for Chapter 
II. Derivation of the algorithm is fully explained by Sage and Melsa 
Cs3D and Meditch CM17. The algorithm is stated here in the manner of 
Sage and Melsa CS3, Chapter 71. 

The system model is given as 

x/k+1> = j>(k+1 , k)_x(k) + £(k)w(k) (1) 

and 

_z(k) = H^(k)_x(k) + v(k) (2) 

where 

x^k) is the state vector 

Mk+1,k) is the state transition matrix 

r(k) is a matrix 

_w(k) is the driving noise vector 
^(k) is the observation vector 
_H{k) is a matrix 

v_(k) is the observation noise vector . 

The assumed prior statistics are given as 
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E[w(k)l = 0 = H[V(k )2 ■ :M > 


r > 


E-Ex-(-oO = £<o) 


(4) 


covfw(k), v;(j)1 = V (k)fi(j-k) 
— — — w 


(5) 


covQ/(k), v_(])3 =V v Ck)6(j-k) 


(6) 


cov[w(k).. vCjT! = covPx(O), w.(k)J 


= cov[x(0), v(k£] - 0. 


(7) 


where 


6 ( J-k) =- 


1. j = k 

0, J * k 


(3) 


These assumptions give the Kalman filter Mgonthm for the estimate, 
x(k), of x/k). Thc cst1mate 1S 

£(k) = *U,k-1)£(k~1) * K(k)j^(k) - y(k)*(k;k-1 >£(k-1 ) 


(9) 


wher e 


K(k) = V (k/k -1 Ml (k) 


M ( k) V (k/k 1 Ml (k) * V (k) 

— — - — — V 

X 


V (k)M T (k)V* 1 (k) 

— — — V/ 


(in) 


X 
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V (k) = Q_ - j£(k)H(k)Tv O/k-1) 

X X 


V (k+1/k) = «(k+1.k)V (k)* T (k+1,k) 
x x 


( 11 ) 


°^OOR L c^t GE 18 
QUAUTXj 


+ Kk)V ] ^(k)r^('k) 


(12) 


x(k) = x_Ck) - x^(k) (V 7 ') 

Tne ease with which this algorithm can be implemented on n digital com- 
puter is evident from the above equations. Sage and Melsa CS?, Chapter 
73 give a good discussion of all the terms used above. 
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Apo.-nJix II 

M-itrix Inversion U'nnn 

The matrix inversion l emmi is included here for reference since it 
is used in some of the derivations in Chapter III. The formulation and 
i^monstration given here is the same as th-.t giv-n by Sage and Melsa 
JSO, d. 499-500] . 

Matrix Inversion Lemma 

If for any N x N nonsingular matrix A and any two N x M matrices 
2 and C_, the two matrices ( A_ + ocj ) and (I_ + cJa_ *B) are nonsingular, 
then the -matrix identity 

(A + BC 1 )" 1 = A" 1 - A _1 f3(I + C T A _1 3) * 1 C T A“ 1 (1) 


is val id . 

Proof 

Define 

2 = A_ + BC 1 . (2) 

Then since by assumption 2 1S nonsingular, we can write 

2~ 1 2 - L~ JL _1 A + • (3) 

Now postmultiply (3) by A to obtain 



Next postmuttiply each side of (-4) "by 3^ to obtain 
-1 -I -1 T -1 

A 8 = A B + D_ M3C ' A. B 

= + ^^2) • 
y ” "f 

Now by assumption ( I_ + C_ A_ 3) is nonsmgulcr. Hence, we can writ 
(5) 

-1 -1 T -1 -1 

d b = a oa + a b) . 

T -1 

Postmul t ipl y by jA to ootain 

D -1 Be T A _1 = A _1 8(I + C T a“ 1 5)” 1 X T a“ 1 


But it is seen from (4) that we can write 


-1 -1 -1 T- -1' 

A ’ - D 1 = D 'nc A 


Hence using (3) in (7) we can write 


A _1 _ A" 1 ” A^H/I + cVA^cV * 

and using (2) we can finally write 

CA + BC T )' -1 ’ = A _1 - A _1 E3'(I' + C T a" 1 'Q) _1 ‘C T A'' 1 


C5 ) 
f ron 

( 6 ) 

(7) 

(3) 

(9) 

( 10 ) 


This is the dfsirfj result. 
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APPLfli) IX III 

COMPUTER PROGRAMS 



TO TOENTIrY 'OOELS OF SPECIAL HANDS MY MaaIMIJM L1>ELHU0O 
TECH il vUE. 


dimension Y(tsno) .pskisoo.ik. ^T oo.jfjj.sRm.jO) ,$no. *o> *zzrso 

40. 30) . Tn- TA < 10 > , RH ( 3 ^ * JO ) . Tn^ TAP (30) * Z ( 30 » 1 ) • »/ ( Hr • 1) .C3RP (3u» i ) . 
AXU500) .0(1500) 

• • Data Hput 

RE40(^.l) N 

1 FORMAT (15) 

RE4DH.2) (0(1) . I = 

2 FORMAT (tJF I 0 .5 ) 


...Ml MUST RE CHANGED TO CHANGE ORDER OF AR SYSTEM 
Ml = l 

WR I TE (16.71) Ml 
71 FORM4T (IS) 

Li-0 

...LI IS The NUMBER OF TRFNn TERMS 
...INITIALIZATION OF ThETA AnD SII.J) 

10 = 2 

-Y(1)=0.0 

00 70 1 = 100 

Y(I)=0(I)-0(M) 

70 CONTINUE 

M=H 1 ♦L 1 
00 10 1 = 10 
THETA (I)=. 10 
DO 11 0 = 1. m 
S (I.J)= 0.0 
ZZT(I ,J) =0.0 
S p ( I i J) =0.0 
ZZTS ( I .0) =0.0 
TOP ( I . J) =0.0 

11 CONTINUE 

sn» r ) = i.o 

10 CONTINUE 

00 12 1=1 ♦ * 

Z(Ii 1)=J.Q 
CORR ( r » 1 ) =0 « 0 
sz u ♦ n =0.0 

12 CONTINUE 

...INITIALIZATION COMPLETE 
...COMPUTATION OF THETA ESTIMATE FOLLOWS 



GO TO 42 
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Aft 


20 


46 

45 

44 

43 


63 

6 ? 

61 

60 


22 

21 


23 


67 

66 

65 


2*- 

13 


CONTINUE 
TF'1 p = 0 .0 
DO 20 1 = 1. M 

7 T $7 = T E i (1 4 1) «bZ ( I * U 
T- Mr-/fs { 

CON T IN Jc 
DE N0*. = 1 • 0 « 7 TSZ 
DO 43 L= l-» * 

DO “4 1=1^ 

7ZTM i ,l>=0.0 

no 4S J-i . * 

IF *777 (I ,N.«,a.u3. -PCJ.L) .5 1*0. 0) GO TO 46 
77TS Il.L)sZ7TSU tL) *77T < I. J>*5?CJ.LI 
GO TO 45 

Z7TSU .L)=Z7T$CT.L>+0*9 

CONTINUE 

CONTI* UE, 

CONTINUE 
DO 60 L=1»M 
DO 61 1= i *h 
TO D ( 1 * L J =0.0 
00 6? J=1. M 

IFfSPdi J> .eo,O.O.Ort.77TS(J^U ♦ E'l.ri.l’l GO TO o3 
TOP II .L) =TOP { I « L) ♦Sr ( I . J > *Z ZT 5 ( J . L 1 
GO TO o? 

TOP (I * L > - T0 P (I »L) ♦ 0 . Q 
CON T IN JE 
CONTIN 
CONTINUE 
00 21 1=1** 

00 22 J- 1 . M 

S{I*J>=3 p (I.J>-( t TOP ( I * J) ) /OENOM) 

CONT l N ' J r 
CONTI iijc 
TCPP=0.0 
00 23 1 = 1*** 

TM7=TC ^♦TH£T4pm«Z(I , n 

TC MP=T47 

CONTINUE 

COEFF^Y K)-TMZ 

DO 65 1= ! * ' 

CORP( I - 1 5 =0 . 0 
00 66 J=l* * 

IFISlI.Jl.t N0.0.^.7to*l),E'J.0*0) GO TO *7 

copp 1 1 * n -Co < i * n*s i 1 * J) *z ( j*o j 

GO TO 06 

CORP ( I * I > =C <jR p ( I * 1 ) *0 . il 
CONT I JUE 
CONT I 4 )Z 
00 24 t=I.M 

THcrAin =TnET4P m ♦corMcid ) ®coeff 

CONT I lUE 
CONTINUE 

* COHPU TATI ON OF THETA ESTI^AT^ COMPLETE 

• COPPUThFIOJ Or fiESIuU»*L VALANCE AND SELECTION CRITERION 


TE*P=0 .0 
= 10 
X (1 )=0%0 
00 25 <=*** W 
X (K-M) -0 .0 
THET A 7 = 0 . G 
00 26 1=1* i\ 

ZU.ll=r u-n 

26 CONTIN Jt 

IF (LI « F ? « 0 ) GO TO 50 
Zt**l )=1 *0 
50 CONTINUE 
00 5? I = U 4 * 

Th£TAZ = ThETA2*T^ETA (I)*Z(M} 

52 CONT I J JL 

R=T ( ( Y CK) -ThETAZ 1 * (Y {*) - THETA Z) J 
TE M F=;ft 

X (fC-MJ - Y IK) -ThFTAZ 
25 CONTI jO t 

RHO=R/ (FLOAT (N-MJ ) 

CRITs-I (FLOAT (MJ/V.u) * VLOG ImhOJ ) -f LOm MM) 

♦ ••RESIDUAL VARIANCE mhI) CRITERION COMPUTATION COMPLETE 


27 

2S 

30 


32 

33 


WRITE (6* *7) 

FOPn AT j 1HI , Jlr-fSTlH^U > COEFFICIENTS 
00 2 l t I - l « 4 

y.pt if i ♦ t-*l r a c n 

r I 1 M 0 • V 1 AC PHA (.1P*3H) = -.E16.AI 
CONTI AJE 

*wn c '( A '*3'U 

AT ( 1-3. * **. “SIUUAL VA~iAnC r - . “ 
*■*! Tt C • 1* I C-M T 

FOA^AT ( tn-J.22-iNFLc.CTI0 I CRITERION = 
CP fc MNJt 

TN=N~“ 

wPircn.-jpi i N 
FOR* AT l 13) 

* w I TE ( -» * « 3 ) lNHti = ur,J 
FORMAT <->F 12 ♦ *>) 

E n 


-LPHMMll 


1 4 . « ) 

* 1 1 6 . d 1 



noo 


- 178 - 


« PHOGR AM TO PtRfORM Z£PO Hf AN VALUATION TEST. 


di^pnsion a<uoo> 

READtM.l, n 
1 F0RM4TUS) 

PfATM U2) (X(]]«I = ), t ) 

? F OP' U( '.Fia.Hj 
T^P-0,0 
00 3 F = 1 1 N 

XMsTFMP** ( [ > 

TfMP=X« 

3 CON UN, !£ 

Xd Ap- ( i , 0 /float (« j > j °x \ 

TfMPso.O 
no A t-l.N 

R = Tf W P* ( X ( I ) -*M4R) * ( x t [ J -XPAR) 
TF 

4 CONTINUE 

pho= ( i .g/float (m- n ) tj,) 

ETA = 3t<RT < r LO*T ( ) /RrtO) 

*RITE<lf>.S> <-Us 

5 FORMAT ( 7HXMAR * ,EIo.H) 

/fRITF (h,M 

S FOR*AT(*jriftHO - » E 1 o M J 
^EUUhWI r,TA 
7 FORMAT ( t>n£ T A s ,tib,e) 

END 


« ? S£^ 


, « « ♦PROGRAM TO PERFORM THE CUMULATIVE PERIQOOGNam VALIDATION TEST, 

DIMENSION X ( \ 3001 ioAMma ( 700 ) »f. i / 0 0 > ♦ oU < 7 0 0 J , *iL ( / 0 0 ) 

READ fM,l) N 

1 FUR'ATU?) 

PEAfH'U*) ( X ( I ) • [ = U'D 

2 FOR’ 1 - T ( *tF 1 2 • o ) 

FN = FLOA r (N ) /?.0 
N?= I F I X (FN> 

DENOMsO,0 

00 3 J-UN2 
SUN 1=3*0 
SUM2=0. 0 

WJ= (6.2B31 *530 7»>FLOmT ( J) ) /FLOAT (N) 

no 4 l=ui 

SUM 1= Ay U ♦ < (L) »COS< 1 AFLOAT (L) ) 

SU w ?rSUM?4X (L) ^SI.UwJ^FLOAT <L> > 

4 CONTINUE 

OAM ^ A ( lU|f?.fl^l|yl)/n OATfN) ) oo?* 1 (P,n<»SUM?) /M OAT (N>) **2 
DEN0 M »0tN , 0H^vA M ' 4 A { J) 

3 CONTINUE 

DO 5 KslPP 
FNU«=0.0 

no b j= uk 

FMJMeFNU<1«GAMMA ( j) 

6 CONTI < J£ 

G(*) =FNJ'VOENO m 

5 CONTINUE 
FLAM=U63 
A = FLAN» 

DO 7 

PytK) =(2,0 >F[_na no/fL')-! (N> ) ♦a/SO^T (float [ J?)) 

bl (Kj = [?,o c flo at (a) /float inii -a/s :=t (float ( \d) \ 

7 CONTINUE 
VfRITF(G^) 

0 FORMAT (IH1,22-CUMUL4TIVC PER 1 ODOOR am ) 

13 FORMAT UHO) 

WR I TF ( 7 * 20 ) N2 

20 FOR 'AT (15) 

00 0 K = 1 »N2 

WK= (6.28 3185307*FLUAT U ) ) /FLOAT (N) 

^PHE(6.10) (X) ,S(K) ♦-UfK) 

10 FO^MUH .** (SXtElb.a) 1 
WRITE(7,2l) *Kt-iL(MtG(fO *5U(KJ 

21 F0R w A T ( **F I 6 . R ) 

IF (G (K) *LT,^L (K) .0R,G (a) ,GT*PU (*<) ) GO TO 11 
GO 10 0 

11 WR I T F ( A * 1 2 ) Wk.k 

12 FORMAT ( IN t?SHDUTSlOE BOUNOARy «AT WK ^ .E16.8*Vri OR K = »I5) 

6 CONTINUE 
END 
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C 

PROGRAM TO M" SERIAL 1 NOt °Er w *i.C£ VALIDATION TEST,* 

CI^MON XllAiO) **2U20) .3(120) 

PE^ n ( •'t n n 

1 FOB ‘AT ( IM 

RFAO ( * .✓) (X([) t I = l,N) 

2 FO«‘«\T (» 

Ft I-* lofLOAT in) 

Nl = !FIX(FMl 
Rl-0.0 
00 3 J-l .N 
R 1 =t> I ♦XUl *<<J) 

3 continue: 

RO-Ri /FLOAT <N1 
00 A K= 1 • N 1 
R2 t K ) “0*0 
K 1 =K ♦ 1 
00 5 J=<1*N 

R2(K) (*s ) -*<J) ^(J-K) 

5 CONTINUE 

R(K) =R2(K1 /FLOAT 

4 CONTINUE; 

SUN- 0 • 0 

00 t K- 1 * N 1 
SU M =SUNtR ( K) (<} 

6 continue: 

ETA- (FLOAT (N-^ U 0 5 Jn ) /{PC;®*?) 
sirens. 7i m*ft- 

7 F 0R M AT ( SHN 1 - *I5t5Xi^)HETA = *E16,P) 

ENO 


C 

c 

c 


.PPOG^H TO CO^RU T t TmF PF?> I OFOG,f .*)* . 


01‘JEr SION < ( 1-00) (700) 

PFAPM,l> n 

1 FOR 'AT (IS) 

RE AOt*', 21 ((ID.hUO 

2 F0^*4T ( -F| 

FN=F ^0*1 ( M J / 2 . 0 

N2=T? r < v 
00 3 J=I,N2 
SU M l-0.0 
Su M 2 = 0 • 0 

oo~l*\-Vn^* i ' 7i *~ l °'' TiJ) } /rLr,AMN) 

Su M i = »l ♦ < (L » *ruv («jof,loa r n m 

* CoJill'OS , ** J ‘ 'J®r l_OA r ‘LI > 

T CONTlNIji* 1 , ** ;> ® Sy ' , l ,/F <- 0A T (M )o„2, ( {c.0»Sy^)/FLOAT{KJ)*»2 

w r i r £ ( ^ ) 

5 FQR *A T ( 1 -*1 . 1 1 rP r ft IujOG*A v ) 

W*ITr (6*6) 

6 FORMAT (L-iOi 
Vf-IT<=:i7.lO) r^2 

10 FGPMTC') 

00 7 I = i...P 

*R[ = (6. 2^31 “v5 "307 ^ -LjjT ( ; ) } /M Oat / rj 

(♦>*-,) 

9 4 1 ? .r 1 -?' 1 : t io.-'.oX, 5 > 

«RITiC7%U> %t, o sk 1 1 1 

11 rOQi\7(2- 16. d) 

7 CONTINUE 
ENO 



nnr> 


-180- 


ORIGINAL PAGE IS 
0F POOR QUALITY 


♦ • ♦ , TO Pi WF OftM TMF CO^ fc LO»a4« VALI>AIt3N TtST. 

0I Mr ‘oK * ALR**4 t<? n t Y t !•. Aft 1 • if t 2 >, 1 *u0 ) . "*YM» ► i CD) . -rf ( I *♦ 0 0 1 ( 2b, 1**0 

AO) * S l » l 1 **00 ) Millay), r ( \<*rjO > 4 f \ /~.t x h',j) 

orn < * , * » -u 

4 FC^ATUSl 

RCAPO. I 1 ( AL^UI l) t 1 = 1 <"l ) 

1 FQP''U <^-i0.p) 

ReAtM^^i * 

2 FOD^ynS) 

PFAO M. 3) (YU) U = 1 »•<! 

3 PllPMAT ( *F 10. -5) 

H-MU1 
H2= H 1* 1 

N 1 =N-1 
MAV r ^ = ?S 

no 10 I v ’=l * *AVG 

DO 11 I = 1«M 
0Y(lMU)=YCI) 

H CONTINUE 

I< = U75321*2M M 
AM=0*0 

V«?FS = ,011-'l ?656 
S-50RT t S) 

DO 12 K=M?.N 

CALL GAUSS! UtS«AM»V) 

(K> =V 
SUM=0.0 
DO 13 1=1. Ml 

SUM = SUM- ALPHA ( I > °U Y UM,K-J) 

13 CONTINUE 

DY ( I H.KJ =SU>1*, (*> 

12 continue 

SUMl=0.0 

00 1A I = 1.N 

SU M 1 =S l J *U *D 7 ( I n 

14 CONT l jjt 

DYBA^(I<M) = (U0/FLOAT (fl) ><>SU M i 
ftl=G.O 

Rl=p}<* (0Y(l*^n-0YbA^ < r n >• OY (I Ul) “OYSlAft ( !.*> 1 

51 CONTINUE 

PG=S 1/FL0AT CM 

no 1=5 2 = 1 «ni 

SUM2=0 • 0 

KN= N-K 

DO 16 J = 1 .KT 

SU*? = $U*<?* COY { JM* j) -DY5AP (IMM« tDY ( I M f J + K ) -*0 Yft Ar< (IH)) 

16 CONT I\U£ 

ft C IM,k ) = ( UO/FLOAT (.«)) «SU“2/^0 

15 CONTI* 'j£ 

10 CONT T lu£ 

SU«J=0.0 
DO 17 1=1 . ( 

SUM3 = S'JM J ♦ Y ( I ) 

17 CONTINUE 

YUAft= ( i 40 /float co j *sum3 
RY 1=0,0 
DO S? 1=1 *N 

ft Y 1 =P Y 1 ♦ C Y ( I ) - YtJA^ ) * ( Y ( 1) - YRA ■< ) 

52 CONTINUE 

ryo=*yi/float <N> 

DO l fl K=l.Nl 

SUM4=0,0 

KN=N*-K 

DO 19 J=I .i<N 

SU M A = SU HA * { Y I J ) - Yt> Aft ) * I Y < J* K ) -Y8 Aft J 

19 CONTINUE 

RY {*) = < 1 .0 /FLO AT («() ) *SUM4/ftY9 

18 CONTINUE 

DO 20 k= 1 • ft 1 
SUM5=0 • 0 
DO 21 J = 1 .mavG 
SUM8=SU«5*R C Jt«) 

21 CONTINUE 

RH(K) = ( 1.0/FLOAT (HmVG) ) "SUMS 

20 CONTINUE 
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DO 22 K = 1 i M 1 
SUN<i = 0. 0 
Do 23 J= ]>• 4A ,/(» 

S JM6=SU 4't* I* ( J.K) ) •*? 

23 COM 1 

SlM*U = >jPT < U.O/r uuaT CMAVO > 

22 COMImOl 
TUIT- ) 

i>: < 6 . io > 

30 FOR AT (1-tl . 12 mC0^KLLL0i»H/^) 

’*«tTr<7.9G) a 

90 FOR' AT < I a 

no 2 ** A^l.Nl 

P (J = 9 %1 ( K J ♦ 1 ^/.^ T iK) > 

RL=m m > 

I~ <*Y <K) «CT. *1i) Co ro rs 
JF(9Y{K).LT* »L > Go r O 3 7 
^witefo*A0) k, <Lii*Y{K » ,mu 

40 FORr , aT<I-».1(‘»** c lt>«to> > 

W»ITE <7.91 > - « -*L ** f CO .5U 

9) FOR*' AT < |:>.3r 1- .rj') 

GO ro Z<* 

25 JRtTE<6.26> f 

26 FOP^AT < l *i » 2 7riOUT 5 1 OF UPPER =CJND AT K 
GO TO 2v 

21 *PITF<6«^> p 

28 FO^AT(H . 27HOUT 5 i jF LURfP ROUND AT K 

29 IOUr=IOJl.l 

24 COMTIp^JL 

IF ( IOuT . r Q. 0 > GO TO 31 
GO TO 33 

31 XPIT5(G,3?1 

32 FO’ 'A T <lirt ,23HPA3ScS CCPPELOORA” TEST 1 

33 CONTI IUE 

DO a 2 K-l , \l 
SU^ 7 = 0*0 
DO 43 

SUm7 = SUM7pDT < I i K ) 

4 3 C ONI IVJt 

YAVO (-<),= (1.0/FLOAT IMAVG) ) ®SUM7 
42 COOT I ,UE 

if«IT c Ui4U 't 
44 FORMAT (IS) 

i/pi re ii>, 4i)'(YAvorni ^c-ivo 1 1 

41 FORMAT (6F 10.5) 

£ ‘lO 


.15) 
« 15) 
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PROGRAM TO COMPUTE AVERAGE (^UTiHLl InFORMA T ION* 


c 

c 

c 


0 I W LNSI0N ALPM W2J) ♦ M-01 .*V| ( >9 .pO! .r./ i --A { :> >%< , ) t L P( ?0 • ?0 1 « mT f ?3 . 

AP{j J ‘ UT. P , > *">l I CP**. *0» • i> • T 1 (2'N /-'ll . 

AGT C P9 . PO J , ^1l t '0,o)»n(^O^U*Tl 

a . /<Ai;r .•nM , w *, c n. a| 

AO^OJ^l (?n,?)l,‘ ' (u * 'G> .*> .1 ii-M-'f..'" , w/tN.fUt^tUnj ,h^ 

a ( i auO) « v'in (po .Pi i •t / rir!c'0,( j in i/^hMIcO^d •> /pcPg.<?'> . vMPO.nn .\ 

. AT ( 20 * 29 1 ♦ ~ 1 . *x(l*»90l .mVh (29, 29) . IhF (H Mi“00) t AvGH’F <b .239 > 

REAL I JFORM.MUU <r 
00 10 1=1^30 
DO 71 J=1.20 
phi t r , j) -o.o 
GAM *A(I«J) = i,0 

hT f 1 ,J) -0.0 

hu. n =o.o 

GPH( I, J' -U . J 

o ; u . j ) - » * j 

on u , j)-c.o 
» aoci . ji 

T 1 (I ,J = 3. 0 

gt ( 1 . j>=:.o 

V*GT { I - J 1 = 0.0 
T? < I , J) =3.0 

GPT t I . J) =n.o 
, VGO { I ♦ J > =9 .0 
1 T3 <I.J1=3.3 
TT < ! . J) =3.0 
VXA U * J) -0*0 
VH(I *J) =3.0 
GA ( I . J» =0 .0 
GAIN < I * J) =0.0 
U<N JJ=C.O 
V A A u (I ♦ Jl =0 . 0 
PH I • J ) = 3 . 0 
P2(I .J> =0.0 
P3( I . J) =0.0 
P< ( I . J> =9 ♦ 0 
VDO tl% J) =0.0 
PHITU • J> =0.0 

vx^Hir ( i . j) =o.o 

PVPU . JHO.o 
VX ( I . J) =3.0 
VT ( I «J1 =0.0 
HVH ( I . J) =0.0 

71 CONTINUE 
70 CONTINUE 

00 7 2 1 = 1 .HA 
AA m =0.0 

«a u j =0.9 

72 cc :r i .Jc 
PfcADU.n N 

l fo^mamisi 


• SYS TE * PARAMETERS FuR Th£ MODEL FOLLO* 

RfAQlP.?) Ml 
FOR'u f 1 lbl 

R£ A D ( 2 • J) (AL^ha U |,Is| *rtl » 

FOR AT ( ^10. S) 

A (]) *A Lhha f 1 j 

A 1 2 1 = 1 « 9*AL^mA ( 2> 

00 A 1 = 1 . M 

A < I ) =AL^-<A ( I ) -AL" N* { l-2> 

CON T IN JK 

A ( P l ♦ 1 ) 2-AL°^C (»M-1 ) 

A (HI *2 ) =-AtPHA (MIJ 


«2 IS THE OPO£P OF TH C STATE E9U4T 1 0 ib 


H 2 =m i ♦ g 

rF( M .FO.fj) GO TO 60 

00 5 1 = 1 . M 


C 

c 

c 



uuouou 


- l'83'l - 


no t> 

Pm i:< i *j> 

6 COMIfJU- 

PrtJ ( 1 vl*l ) = v l.#0 
5 COMIVJl 

no 7 5 J=1.‘**2 

oh !*{'•?, J) = A < O-'J) 

7 CONTI 4UL 
GO TO *3 1 

60 PhK< 1 , 1 )=ALPHA>(1) 
b'l COM TIN j£ 

DO ^ I“l*^ 

GA»* *4 < Im*! 1 ! =0.*0 
A CONTINUE 

10 =l,.l) 

HT { #< ?«-10 =1 »*0 

IF C l.FO.O) GO TO 62 

00 I =1 , 4 
HT ( I ,*1)'=0.0* 

9 CONTINUE 

6 ? continue 

V«-*001^^ l lo6<»0 
00 10 l = b* '2 

00 11 J=1*M2 
VXOU.J) = 0.0 

U, continue 
VXO (lill-l.o 
ro con tin jf , 

VVO=,OOOOOl 

DO 200> ! UT='lo 
VVO=-VVO*10.-'> 
rfPIT c ,(l-3. ?02I* M UT* 

20? FQ=?^4T»( [=ij 

no 1 2 1 1 = 1 *. N* 

vvmswo- 

1.2 CONTINUE 
VhO- i/»/ 0 
VR=tfV ( 1 ) 

vvw=o,*o 

»'• **. ♦End of srsTE” papa*ete*s 
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13 




4'2 


L v a 

17 


no n i=i * *2 

0P( 1«*>1 J =oA^A ( I ,»l$) l^O/VVO) 

CONTlN'J r 

CALL **WTPP (HT ,2J ,>20 .h> 

call patmul: .20 .-^ph) 

Call MtfrsUrTIPV* ->0 « 20,0 \ K b 

CALL MA‘TT^p (OP*? J.? J .~I T ) 

CALL HA<T4ML < 7*0. 01 T . .r20*^t) . 700)’ 

CALL "AT^i,L fPl,.Vn0.2 j >•-71*/ 

CALL MA^TNP'l ~ AMMA » £ j . 2 1 » G T ) 

DO *0 J=1 J .M? 

V«GT M . JJr=V//°GT (il*jr 

CONTINUE 

CALL MATMijL < Gamma 1 , y„ r tT L«20,* 20. 20^T2) 
CALL lATTPP(C,P,20,2j.rPTO. 

00 A1 T = lN‘i? 

00 42 v J=K*2 

VGO'f P.rJ) =VVO^GPT*f Im J> 

CONT IMUE 
CONTINUE. 

CALL ^AT HUL (C'p • VG?. 2 WO . ?0ttT3) 

CALL MtHOOMI .^^j.PnTTI 
CALL -WTVJi'l n . 7XF) 

JJ=i 

00 17' I=iH^2 
00 IB ,J = JJ. >2 
VXA T ^ M . J) = 7*Y M « J ) 

VXA^ ( J « i ) =vaaN I , Jj, 

CGNTtNJc 

VXAP ( I J i ni»VXAf(IvW 

jj=jj* r 
continue 
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c 

c 

c 
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CALL HJL<V' . ^n.PO, ?o.v i) 

rc^*- - i , i 1 > *v / n j j 

o5 u u* i i-r , M^ v,A ,,Mf ’ ') 

14 

no i a j = i 
po h j=l - 42 
u r t * j i = j , j 
16 COST l \ J- 

uu«n=Ko 

15 COS T [ 

Call r -IJL 

CALL w*TVJ JClj.Pl 

call wa Tf'UL (P^WX«K,?0.^,20,P3J 
oo~i9 i=i ,m? 

OO PO J= J J . h? 

PA (I . J»=h^(I * j) 

20 COMTIhUt 

pxd *n=P3cr, l ) 

JJrJjr] 

19 COMTI m«J£ 

CALL MA T T3p (P hI . l ) .£ i , phjt > 

CALL *4 A T -njL t V / 9 • ] 7 /xPHTT) 

call wa t -mil < • v^p-i f r*2o.?o»2<),pvp] 

call MATADO(P V P,T<.2o!20,vT> 

00 31 1=1, M5 
DO 3P J*JJ ,v 
vxci t j)=tfT(r 
U(J, I ) = V T ( I , J ) 

32 COfin.Ut 

v* ( i.n = ;T(r«n 

31 COHUN’Jc 


OF I JITULI7ATIO j 


.COMMUTATION OF VA^J^NCcS FQLLO/S 


^UTlMr=O.Q 

00 loo 

V9 = VV f ft ) 

00 22 1=1, M ? 

2? CONTI \ jP 34 ' MA f, « 11 ° N . O/'/V t<) 1 

CALL vatmijl (OS>,d,?o^0,20.-P«i 
CALL HA f [PHI , I, pj,, »*; *>-{ * .» i , 

CALL ^Arwcoi.p u>*Ni ri ‘ l} 

S*}*1 J 'UL «^1T 9 c t • 9 )*?■»• yi»0 ) 

M £ £ ,f W- IP 1 * </ov t ^ J . -m) , pij » T 1 ) 

00 43 

4 3 

23 CO'ITJVJP 

C4LL MarwiJl. (GP,^r)0,cD,>£),;4.r3) 
C-LL M4T5^rTr.T3 (< ;0.23.Vx4l 

D0~^ l = l .M S 
00 2S J=JJ*^2 
VX AP [ I , J) sy / A U , J) 

25 co\n^A i ^ KXtl ’ 

^I; ns ' XAn - n 

24 CONTIf^jr: 

call ^AT MFL [/XAP,nT,>0,20 .Pft.WH) 
CAUL ^TM ULr HWf.,2j;pn*20^vii 1 

g£ L ^,' |ITV m 2 VXA!> ’' ,T ’ ?0,20,? 0* C4 ' 


6" 


4 



ono nonoonn 
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GAlMM) = (I'l) 

26 COMTih 

CAUL ma r^UL IGA IN. n * 20 • PO * ?0 . ) 

CALL 

CAUL MATMUL (-V.VAAP.2C.20.20.pl ) 
JJ = 1 

Dn 37 r=N«? 

00 ?M J=JJ* V 
Pa (I t J) ^**(1 . JJ 
Px l J. 

2fl CONTINUE 
PX ( I . I) 

JJ=JU* 1 

27 CONTI .U£ 

IJ = 0 

11 = 1, 

OU 23 J=1.M2 
OC 30 1 = 1. II 

u=r j* i 

4 Aft U J>=P<II.J) 

30 CONTr.j- 

ii = hm 

29 CONTINUE 

. ,.EnO OF /ft^flANCE CONFUTATION 


MUTUAL INFORMATION COMPUTATION F0LL0rt5 


MUTINFsMUTINF ♦ l.5*jApJ(M?*in 
AVGP.F ( *' J T ♦ * ) -MUT I .*r 

..•.MUTUAL INFORMATION* COMPLETE 

100 continue 

200 CONTIN'J* 

DO 201 N=?»N 

WRITE (6, 5?) K.ftVGINFU.K) , AVGINF(2-;K) , A7GlNF(3,fO * AVG I NF <4.K) .AVGI 
ANF fa.*) 

52 FO Dm U(Ih .I5.5F12.S) 

WRITER 7.203) rU. AvGI >F U . <) . AvGlNF (2.M . AVGlNF ( j r*> . A VO INF (4,0 « AVG 
AInF(o.K) 

203 FO^A M I5.5F12.5) 

2G‘l CONTIWJt 
END 
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*U*«0UT IN r “U "iL U i ♦' * il 1 

OI^LNs [G\j * (pOiPJi * .{?'.*PONCt^ 0 . 20 i 
DtJ 1 j£l,Nl 
DC d I“UM 
C li * J> = 0.0 
DO 3 *= 1 .M? 

C (I < J» =Ct I f J> *A (I ,M J> 

3 f'OMT l Mr. 

£ CO*»T P.'Jf 
1 CONTlMJd 

hetihn 

EMD 


? 

1 


SV? k,o yi l * ,r 

no •ri^o«20) 

Do d J- l Jfip 

c f r * ji = a ( r . jj -►! ( i , 

CONTI j.jt 
CONT \\ )t_ 

*t TU-TI 


u) 


2 

l 


SU^°OiJT I n£ H£T ADOC A.^*M «» p.r J 
Dt«t^m A |P 0 *? 0 J t j t? J • ✓ o i *C <20 »20 ) 
DO 1 iM.Ni 

oo a j= i .np 

C < l * j > = ' 

CONTINUE 

CO r lT 

TlMil 

£nO 


sua^ouT i n^ mat (ioiii ♦ ip ♦ h » 
n i«* ns t )'i AUcOt2‘j»^(>’J.; , )i 
00 1 I - t « M • 1 
no ^ j= i . i 

CO‘U T < i . 

CO 1 Tt » Jz 
R* TU^ i 
END 




